Distributed Databases

Introduction

A distributed database spreads data and compute across many machines — often many regions — to deliver scale, availability and low latency that no single machine can. MongoDB is distributed by design: replica sets for HA, sharded clusters for horizontal scale, and Atlas Global Clusters for geo-distribution.

The trade-off is governed by the CAP theorem (Consistency, Availability, Partition tolerance — pick two) and the PACELC extension. MongoDB tunes the dial per-operation via write/read concerns: you choose strict consistency or lower latency on every call.

Understanding the topic

Distribution primitives in MongoDB:

• Replica set — same data on multiple nodes. Solves availability & durability.
• Sharded cluster — data partitioned across shards by a shard key. Solves scale.
• mongos router — stateless query router that hides shards from applications.
• Config servers — store cluster metadata (chunk → shard mapping).
• Zone sharding / Global Clusters — pin data to specific regions for residency & latency.
• Causal consistency sessions — read-your-writes guarantees across nodes.

Syntax reference

Sharded cluster anatomy:

┌────────────────┐   ┌────────────────┐
│ Application(s) │ → │  mongos router │ ← stateless, scale horizontally
└────────────────┘   └───────┬────────┘
                             │ routes by shard key
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
   ┌──────────┐        ┌──────────┐         ┌──────────┐
   │ Shard A  │        │ Shard B  │         │ Shard C  │
   │ (rs0)    │        │ (rs1)    │         │ (rs2)    │
   └──────────┘        └──────────┘         └──────────┘
                             ▲
                             │
                     ┌──────────────┐
                     │ Config RS    │ ← chunk map, balancer state
                     └──────────────┘

Real-world use

Coinbase, Sega, Toyota and SAP run sharded MongoDB clusters with hundreds of shards serving petabytes of data. Atlas Global Clusters enable GDPR data residency: EU users' data physically stays in EU shards, US users' in US shards — transparent to the application.

Best practices

• Pick the shard key like you pick a primary key — it's nearly impossible to change later.
• Avoid monotonic shard keys (timestamps) — hot-spotting kills throughput.
• Use hashed shard keys for write distribution, ranged for locality queries.
• Start without sharding; add it only when a single replica set hits limits.

Common mistakes

• Sharding too early — operational complexity dwarfs the benefit for sub-TB workloads.
• Forgetting that secondary indexes on non-shard-key fields require scatter-gather queries.
• Ignoring chunk balancer alerts — imbalanced shards mean uneven CPU & disk.