Production Optimization

Introduction

Production optimization is the discipline of making a working cluster fast, predictable, and cheap under real load. It's the difference between a demo that works for 10 users and a system that serves 10 million requests an hour with a 50ms p99.

It combines schema, indexes, connection pooling, query shape, hardware tier and replica topology. Every optimization should be measured before-and-after with Atlas Performance Advisor, db.currentOp() and explain("executionStats") — never guessed.

Understanding the topic

The production optimization checklist:

• Schema first — bad models cannot be fixed by indexes; embed read-together data, reference write-heavy data.
• Index every query. Every find, $match and sort in production code must hit an index — verify with explain().
• Right-size the pool. Default pool 100 is wrong for serverless (use 10) and wrong for batch jobs (use 200+).
• Use projections. Never find({}) without trimming fields — network and BSON parsing dominate latency.
• Pin reads strategically. Analytics → secondary, transactions → primary, geo-distributed apps → nearest.
• Cache the hot 10%. Redis or Atlas in-memory storage in front of catalog/profile reads.

Syntax reference

Production tuning loop:

1. Capture   → Atlas Profiler / db.system.profile / slow query log
2. Analyze   → explain("executionStats")   →   COLLSCAN? IXSCAN? sort stage?
3. Hypothesize → schema change | new index | rewrite pipeline | scale up
4. Apply on staging → run k6/locust load test → compare p50/p95/p99
5. Roll out with monitoring  →  observe for 24h  →  document the win

Informative example

Before/after of a slow checkout query:

// BEFORE: 1.8s on 50M-document collection
db.orders.find({ userId, status: "open" }).sort({ createdAt: -1 }).limit(20);
//  → COLLSCAN on userId, in-memory SORT — Atlas alerted at 2s SLA.

// AFTER: 4ms
db.orders.createIndex({ userId: 1, status: 1, createdAt: -1 });   // compound, sort-ordered
db.orders.find({ userId, status: "open" })
  .sort({ createdAt: -1 })
  .limit(20)
  .hint({ userId: 1, status: 1, createdAt: -1 })
  .project({ items: 1, total: 1, createdAt: 1 });

Real-world use

Toyota's connected-car platform cut p99 from 2.3s to 60ms by adding three compound indexes, switching analytics reads to dedicated nodes, and converting telemetry to time-series collections. Total infra cost dropped 35% in the same quarter.

Best practices

• Always benchmark with realistic data volume — performance bugs hide until 10M+ documents.
• Enable Atlas Performance Advisor — it suggests indexes with measured impact.
• Set maxTimeMS on every user-facing query to fail fast instead of bringing the cluster down.
• Run chaos drills: kill a secondary, fail over the primary, verify SLA still holds.

Common mistakes

• Adding too many indexes — every write pays the cost; review with $indexStats quarterly.
• Over-embedding — documents > 16 MB will hit the BSON limit; split before you hit the wall.
• Trusting averages — always look at p95/p99, not mean latency.