Capacity Estimation and Workload Modeling

Advanced System Design & Software Architecture · Lesson 24 · High-Level Design and Distributed Systems

Lesson Overview

During enrollment week, TechLearningPro's classroom team sized the video tier as if every registered learner pressed play at once. The number that came back was several terabits per second, and the sketch that followed had a custom edge network the company could not operate. A second person divided the same catalog by "a few big servers" and called it done. Both errors are the same error: a workload model that never names who is active, for how long, at what bitrate, and in which hour. The first model treats a registration row as a concurrent stream. The second treats a year of bytes as a feeling.

Capacity estimation is a model that turns labeled assumptions into rates and bytes a design can accept or refuse. The outputs that matter in this course are requests per second, concurrent work, storage growth, and bandwidth, each with a peak shape and a retention rule. The output that does not matter is a false sense of precision. Two significant figures and an assumption table will delete the wrong boxes. Six significant figures copied from a spreadsheet will not.

/learn/high-level-design/capacity and /learn/system-design/back-of-envelope-estimation introduce the arithmetic. This lesson stays on the arithmetic long enough to separate a steady evening from a launch window, to apply Little's law without pretending it is a queueing formula, and to record the envelope as a decision. Lesson 23 used a one-row estimate to keep a permalink design small. Here the worked system is TechLearningPro's classroom: progress heartbeats, catalog reads, and video bytes. Those three workloads saturate different resources. A single QPS figure would have hidden that.

You will recompute every figure in this lesson from the assumption table. If an input changes, the architecture conclusion has to be allowed to change with it.

Learning Objectives

By the end of this lesson you will be able to:

1. Build a workload model from registered users, daily active users, actions per user, and a stated peak shape.
2. Convert daily events into average and peak request rates, and cross-check a peak factor against an hour-share.
3. Estimate concurrent viewers with Little's law and turn concurrency plus bitrate into egress.
4. Estimate storage two ways: a steady state under retention, and cumulative growth when bytes are not deleted.
5. Apply replication and index overhead only where you actually operate the copy, and say when a cloud object's durability is already in the service.
6. Show that two workloads in one product can imply different designs, and write the envelope as a decision with a reopen condition.
7. Spot an estimate that uses registered users as concurrency, or that sizes an application tier for video bytes.

Prerequisites

• Lesson 23's sequence. Estimates are step 3 of a design, and they are allowed to send you back to the scenarios.
• Arithmetic with units: bytes versus bits, seconds per day (86,400), and the difference between MB/s and Mb/s. This lesson uses decimal units: 1 GB = 10^9 bytes, 1 TB = 10^12 bytes. Where a vendor quotes tebibytes, convert before comparing.
• Lesson 2's habit of labeling a measure. An estimate is an assumption until a counter replaces it.
• No queueing-theory course. Little's law is stated in full. The knee of a latency curve is lesson 25.
• A JDK is enough for the worked class. There is no storage API to call.

Detailed Core Concepts

What an estimate is for

An estimate answers a design question. "Do progress writes require a sharded store?" "Must video bypass the application process?" "Is a 10-minute launch a different shape from Thursday evening?" If the number cannot change a decision, skip it and spend the time on a number that can. Permalink storage in lesson 23 was worth one line because it killed a shard. Video egress here is worth a page because it kills an architecture.

The estimate is not a capacity plan for purchasing. Purchasing needs benchmarks, headroom policy, and a forecast the finance team will sign. Back-of-the-envelope work needs honest inputs, visible arithmetic, and a conclusion in words: which tier is bored, which tier is the constraint, and which input, if wrong by 10×, would change the conclusion. That last sentence is the sensitivity. Write it down. It is the difference between a model and a decoration.

The funnel from registered users to a rate

Start with the population that exists, then narrow it.

Registered users are rows. They cost storage if you keep them, and they cost nothing in CPU while they sleep. Using them as concurrency is the terabit mistake.

Daily active users (DAU) are people who do the action at least once in a day. DAU is a behavior metric. A product with 2,000,000 registrations and a 10 percent DAU ratio has 200,000 people in the model. The ratio is an assumption until analytics owns it. Write DAU = registered × ratio so a change in either input is obvious.

Actions per active user turn people into events. A learner who watches for 30 minutes and whose player reports position every 15 seconds produces 120 heartbeats that day. A learner who opens 15 lesson pages produces 15 reads. These are different actions and become different rows in the model. Averaging them into "requests" hides the write path inside the read path.

Daily events are DAU × actions per DAU. Average rate divides by 86,400 seconds. The division is the step people do in their head and round badly. Show it.

Peak shape turns an average into the rate you design for. Two simple models are enough for this lesson:

• A peak-to-average factor. If the evening is four times the daily mean, peak QPS is average × 4. The factor is an assumption about the day, not a law.
• An hour share. If the busiest hour carries 15 percent of the day's events, peak-hour QPS is daily × 0.15 / 3,600. This model cannot see a 10-minute spike inside that hour. When the product has a launch minute, replace the hour with the real window: fraction × daily / windowSeconds.

Use both on the same workload when you can. If they land in the same order of magnitude, the shape is robust. If a launch window is ten times the evening model, you have two envelopes, not a disagreement to average away.

Headroom is a policy, not a measurement. A factor of two on the design peak covers a wrong DAU ratio and a noisy neighbor better than it covers a forgotten workload. Headroom does not replace a missing action. Doubling a heartbeat estimate does not size the catalog launch.

Read mix, write mix, and payload

QPS without a mix invites the wrong store. Heartbeats are writes of a small row. Catalog pages are reads of a larger document. Video is neither: it is a byte stream that should not appear as application QPS at all. Report three lines:

• Read QPS and the response size you assume.
• Write QPS and the row size you assume.
• Stream concurrency and bitrate, if bytes dominate.

API bandwidth is then QPS × bytes on the wire. For heartbeats of a few hundred bytes, this product is usually boring next to video, and the boredom is a finding. It tells you not to optimize the application network for the player. It does not tell you the database is bored. Database cost follows writes, indexes, and fsync behavior, which lesson 28 measures. Here you only reserve the right order of write rate so you do not propose a shard for 1,000 upserts per second.

Concurrency and Little's law

Little's law says that in a stable system the mean number of items in the system equals the mean arrival rate times the mean time each item spends in the system: L = λ × W. It does not assume a particular arrival distribution. It does assume you are looking at a system that is not growing without bound during the window you average over. A launch that piles up retries forever is not stable, and the law will not rescue the estimate.

For video, define an item as one in-progress watch. If watches begin at rate λ per second and last W seconds, the mean number of concurrent viewers is λ × W. You often do not know λ directly. You know person-seconds of watch time in a day. Those two views match:

concurrent average = (DAU × watch seconds per DAU) / 86,400

That is Little's law with the day's arrivals spread across the day. Multiply by the same peak factor you used for QPS only if watch time peaks with the same shape as heartbeats. It usually does, because heartbeats are emitted by those same watches. Say so. If catalog browsing peaks in a launch hour when nobody is watching yet, give the catalog its own factor. One factor for the whole company is how the video tier inherits a sale spike it does not have, or how the sale API inherits a video plan it does not need.

Concurrency is the input to bandwidth:

bits per second = concurrent viewers × bitrate per viewer

Divide by eight for bytes per second. Keep bits in the sentence until the last step. A 2.5 Mb/s stream is 2.5 megabits, not 2.5 megabytes. The factor-of-eight mistake is an architecture-changing mistake: it makes the network look eight times cheaper or eight times more expensive, and someone will buy the wrong circuit.

The bitrate itself is an assumption about the rendition mix. A classroom that serves mostly 720p does not have the same egress as one that defaults to 1080p. Record the mix. When in doubt, bias the design bitrate upward and say you did. A sensitivity note — "if the average rises from 2.5 to 5 Mb/s, peak egress doubles and the origin-miss budget doubles" — is more useful than a false single number.

Storage: flow, stock, retention, copies

Storage estimates fail when flow and stock are mixed.

Flow is bytes per day you write. Stock is bytes you keep. If you delete or roll up data after R days, and the flow is steady, the stock approaches flow × R. That is the retention formula. Heartbeat events kept for 30 days and then dropped occupy about a month of flow, not a year.

If you do not delete, there is no steady stock. Course video retained while the course is listed accumulates. The one-year stock is the sum of a year of ingest, plus whatever you already had. Quote both the annual ingest and the horizon you were asked about. "We need 23 TB" is meaningless until the reader knows whether that is next year or the steady state.

What a byte contains must be stated. A progress row of 80 bytes of payload can be 256 bytes once you count tuple overhead and the indexes you will actually build. Put the overhead in the assumption. Do not apply a mysterious 3× at the end and also inflate the row.

Copies you operate multiply stock. One primary and one replica is about 2× disk for that store, and writes are applied on both, so write I/O follows the copy count. A read replica you serve traffic from is still a full copy.

Copies the provider operates are different. Object storage sold with eleven nines of durability already includes the provider's replication. Multiplying your object-storage bill by three "because HDFS used three copies" double-counts, unless you pay for a second region or a second copy you control. The decision record should say which one you mean. A cross-region copy for disaster recovery is a real multiplier and a real bill. It belongs to the recovery scenario, which lesson 64 designs. It is not an automatic ×3 on every diagram.

Indexes and write amplification affect I/O more than they affect the envelope's QPS. An upsert that maintains two indexes is more write work than an append of the same payload. For the envelope, count the logical write QPS, note the index count, and refuse to invent a precise WAL multiplier. Measure that on the engine you chose. The envelope's job is to show whether logical writes are hundreds per second or hundreds of thousands.

Bandwidth is a path, not a single pipe

Draw the path the bytes take, then estimate each hop. For classroom video the path is: player, edge cache, origin (object storage or a fill tier), and nowhere near the Spring Boot process. A 5 percent origin-miss assumption means the origin sees 5 percent of the edge bitrate, plus whatever the misses concentrate on a new rendition. The 5 percent is an assumption with a huge sensitivity. A bad cache key that misses half the time turns a comfortable origin into the incident. Lesson 26 and lesson 27 own the mechanisms. This lesson owns the number that makes those mechanisms mandatory.

Application-tier bandwidth still matters for APIs. Compute it so you can show it is small, and so a later feature that starts proxying video through the API is visible as a hundred-fold jump rather than as a surprise invoice.

Two envelopes, because the day has two shapes

TechLearningPro's Thursday has a broad evening hump of watching. Enrollment morning has a narrow hump of catalog reads and checkouts, and watching may even dip while people browse. A model with one peak factor will size one of those days wrong. Write two envelopes when the shapes differ, and attach each one to the tier it stresses. The video edge is sized from the watch envelope. The catalog read path is sized from the launch envelope. The progress table is sized from heartbeat writes, which follow watches, not from catalog QPS.

This is the point the primers can leave implicit. A single back-of-the-envelope "QPS for the system" is how a team shards the database that was bored and leaves the edge unplanned.

Sensitivity, headroom, and what would change our mind

For each envelope, name the input that the conclusion depends on. For video, the conclusion "bytes do not enter the application" survives even if every input is off by 10× in the friendly direction: the bitrate would have to collapse implausibly far before a JVM should stream the lessons. For progress storage, the conclusion "one primary is enough for a year of upserts" does not survive a product change that stores every heartbeat as an immutable event for a year inside that same primary. The reopen condition is a workload change, not a slogan about scale.

Headroom of 2× on peak QPS is the policy used in the worked example. It is not magic. It covers a DAU ratio of 10 percent that is really 15, and a peak factor of 4 that is really 6. It does not cover treating events as full history in the OLTP store. Policy factors multiply a finished model. They do not repair a wrong one.

What this lesson will not pretend to compute

Disk IOPS limits, JVM GC cost, connection-pool math, and the latency knee at high utilization are lesson 25 and lesson 28. An estimate can say "about 1,100 writes per second of small upserts" and stop before claiming a core count. Claiming a core count without a benchmark is how the envelope becomes fiction in the other direction: false precision. Say "this rate is within what a single modest primary is commonly asked to do for small upserts; confirm with a benchmark before you buy a shard." That sentence is technically honest.

Architecture and Design Diagrams

The model is a funnel. Registered users enter at the top. Only concurrent viewers reach the bandwidth box. Only retained bytes reach the storage box. Launch-day catalog reads never enter the video funnel.

mermaidCopy
flowchart TD
    registered[Registered learners]
    dau[Daily active users]
    actions[Actions per user by type]
    daily[Daily events]
    avg[Average rate over 86400 s]
    peak[Peak by factor or by window]
    split{Which resource?}
    qps[API read or write QPS]
    concurrent[Concurrent viewers via Little]
    bytes[Egress bits per second]
    stock[Storage stock and retention]

    registered --> dau
    dau --> actions
    actions --> daily
    daily --> avg
    avg --> peak
    peak --> split
    split --> qps
    split --> concurrent
    concurrent --> bytes
    daily --> stock

The runtime path shows why the funnels must stay separate. Heartbeats and catalog reads terminate at the application and the OLTP store. Video bytes terminate at the edge.

mermaidCopy
flowchart LR
    player[Player]
    edge[Edge cache]
    origin[(Object storage)]
    api[Classroom API]
    db[(Progress and catalog OLTP)]

    player -->|video bytes| edge
    edge -->|misses only| origin
    player -->|heartbeat and page reads| api
    api --> db

A launch morning and a watch evening are different load shapes on those arrows. The diagram is boring on purpose. The numbers in the next section are the design.

Technical Explanation

Worked assumptions for the classroom

These inputs are assumptions. They are the kind of numbers a staff engineer writes down when analytics is not yet in the room, so that analytics can replace a row without rewriting the method.

Heartbeats and catalog reads on a watch day

Heartbeats per DAU = 1,800 / 15 = 120.

Daily heartbeats = 200,000 × 120 = 24,000,000.

Average rate = 24,000,000 / 86,400 = 278 QPS.

Peak by factor = 278 × 4 = 1,110 QPS.

Peak by hour share = 24,000,000 × 0.15 / 3,600 = 1,000 QPS.

The two shapes agree. Design peak is 1,100 QPS, and the headroom policy asks the write path to tolerate about 2,200 QPS without a redesign. That is a benchmark target, not a shard key.

Catalog reads on the same day: 200,000 × 15 = 3,000,000 daily reads, 35 QPS average, about 140 QPS at a factor of four. They are a rounding error next to heartbeats in rate, and they may still dominate read latency if each read is a heavier query. Rate is not cost. Say which one you mean.

Wire size for a heartbeat, request plus response, assumed 600 bytes: 1,110 × 600 ≈ 0.67 MB/s at the peak. The application NIC is not the constraint.

Concurrent viewers and egress

Daily watch-seconds = 200,000 × 1,800 = 3.6 × 10^8.

Average concurrency = 3.6 × 10^8 / 86,400 = 4,170 viewers.

Peak concurrency at the same factor of four = 16,700 viewers.

Peak egress = 16,700 × 2.5 Mb/s ≈ 42 Gb/s, which is about 5.2 GB/s.

Origin at a 5 percent miss ratio ≈ 2.1 Gb/s, plus a concentration risk when a new lesson misses everywhere at once. Even the miss path is a dedicated origin problem. Forty-two gigabits per second through the classroom JVM is not a tuning exercise. It is the wrong path. ADR-103 below records that.

The false model, for the file: 2,000,000 registrations × 2.5 Mb/s = 5 Tb/s. It is arithmetically tidy and operationally meaningless. It counts people who are not watching. The funnel exists to prevent that multiplication.

Progress storage, both ways

If someone inserts every heartbeat: 24,000,000 × 256 bytes ≈ 6.1 GB/day, about 2.2 TB/year, before the replica. Thirty-day retention of that stream approaches 30 × 6.1 GB ≈ 180 GB of stock, then stops growing, if the delete job actually runs. A year of undeleted events on the OLTP primary is a product decision in disguise.

The design we accept stores the latest position per learner and lesson. Touches per day = 200,000 × 3 = 600,000. New pairs at 20 percent = 120,000 rows/day. At 256 bytes that is about 31 MB/day, about 11 GB/year. With the replica, about 22 GB/year of disk we operate. That number cannot justify a sharded progress store. It can justify a careful upsert and a benchmark at 2,200 QPS. Raw heartbeat events, if product analytics wants them, go to object storage or a warehouse with a 30-day or 90-day retention, in a separate flow. They do not join the OLTP stock.

Video stock

Mezzanine at 15 Mb/s: 15 × 10^6 / 8 × 3,600 = 6.75 GB per hour of timeline.

Forty hours a week ≈ 270 GB/week ≈ 14 TB in a year.

Renditions at 9.3 Mb/s combined ≈ 4.2 GB per hour ≈ 8.7 TB in a year.

Annual ingest ≈ 23 TB, and it accumulates while courses stay listed. There is no retention steady state unless unpublishing deletes bytes. The envelope assumes delete 90 days after unpublish, and assumes unpublish is rare in year one, so year-one stock is about a year of ingest. Provider durability is not an extra ×3. A second region we pay for would be an extra copy, and it is not in this envelope. Lesson 64 can add it with its own scenario.

Launch-day catalog, a different shape

Launch DAU 400,000 × 40 reads = 16,000,000 catalog reads that day.

Forty percent in two hours: 6,400,000 / 7,200 ≈ 890 QPS as the window's mean.

A burst of three inside that window ≈ 2,700 QPS of catalog reads.

Heartbeat math does not see this number, and this number does not size the video edge. The catalog envelope is what tells you the read path, and any cache in front of it, must be reviewed against a launch. Lesson 27 will be judged by this peak, not by Thursday's 140 QPS.

Checkout writes stay small in this model: even tens of thousands of enrollments in the window are single-digit or low tens of QPS. The launch is a read problem with a strict correctness problem on the few writes that take a seat. Do not size the write store as if it were the read QPS. Do not cache the seat count to survive the read QPS. That second warning is a capacity conclusion with a consistency tax, and lesson 27 will have to live with it.

Sensitivity

Practical Implementation or Design Walkthrough

Problem statement

Produce an envelope the classroom squad can put in a decision record. The envelope must recompute from code so a changed DAU ratio cannot drift across a slide and a spreadsheet.

The class

The class encodes the formulas and nothing else. It does not know about HTTP, disks, or video codecs. Callers pass the assumptions. Tests pin the worked example so a "cleanup" of the arithmetic fails in CI.

javaCopy
package com.techlearningpro.capacity;

public final class WorkloadMath {

    public static final double SECONDS_PER_DAY = 86_400d;
    public static final double SECONDS_PER_HOUR = 3_600d;

    private WorkloadMath() {}

    public static double dailyActiveUsers(double registered, double dauRatio) {
        requireNonNegative(registered, "registered");
        requireUnitInterval(dauRatio, "dauRatio");
        return registered * dauRatio;
    }

    public static double averageQps(double dailyEvents) {
        requireNonNegative(dailyEvents, "dailyEvents");
        return dailyEvents / SECONDS_PER_DAY;
    }

    /** Peak from a multiple of the daily mean. factor of 4 means the peak is four times the average. */
    public static double peakFromFactor(double averageQps, double peakToAverage) {
        if (peakToAverage < 1d) {
            throw new IllegalArgumentException("peakToAverage must be >= 1");
        }
        return averageQps * peakToAverage;
    }

    /** Peak-hour mean when the busiest hour holds the given fraction of the day. */
    public static double peakHourQps(double dailyEvents, double fractionInPeakHour) {
        requireUnitInterval(fractionInPeakHour, "fractionInPeakHour");
        return dailyEvents * fractionInPeakHour / SECONDS_PER_HOUR;
    }

    /** Mean items in system. Stable window only. */
    public static double littlesLaw(double arrivalRatePerSecond, double residenceSeconds) {
        requireNonNegative(arrivalRatePerSecond, "arrivalRatePerSecond");
        requireNonNegative(residenceSeconds, "residenceSeconds");
        return arrivalRatePerSecond * residenceSeconds;
    }

    public static double averageConcurrency(double dailyPersonSeconds) {
        return averageQps(dailyPersonSeconds);
    }

    public static double megabitsPerSecond(double concurrent, double megabitsEach) {
        requireNonNegative(concurrent, "concurrent");
        requireNonNegative(megabitsEach, "megabitsEach");
        return concurrent * megabitsEach;
    }

    /** Steady stock when flow is constant and data older than retentionDays is deleted. */
    public static double steadyStockBytes(double bytesPerDay, double retentionDays) {
        requireNonNegative(bytesPerDay, "bytesPerDay");
        requireNonNegative(retentionDays, "retentionDays");
        return bytesPerDay * retentionDays;
    }

    private static void requireNonNegative(double value, String name) {
        if (Double.isNaN(value) || value < 0d) {
            throw new IllegalArgumentException(name);
        }
    }

    private static void requireUnitInterval(double value, String name) {
        if (Double.isNaN(value) || value < 0d || value > 1d) {
            throw new IllegalArgumentException(name);
        }
    }
}

averageConcurrency divides person-seconds by seconds in the day, which is the same operation as a rate. Naming it twice keeps call sites honest about units. littlesLaw is the arrival form. They meet when dailyPersonSeconds = arrivalRate × residence × 86,400 and the arrivals are spread over the day. The test uses the person-second form because that is the input the product actually has.

Pinning the worked numbers

javaCopy
package com.techlearningpro.capacity;

import static org.junit.jupiter.api.Assertions.assertEquals;

import org.junit.jupiter.api.Test;

class ClassroomEnvelopeTest {

    @Test
    void watchDayHeartbeatsAndViewers() {
        double dau = WorkloadMath.dailyActiveUsers(2_000_000d, 0.10d);
        double heartbeats = dau * (1_800d / 15d);
        double average = WorkloadMath.averageQps(heartbeats);
        assertEquals(278d, average, 1d);
        assertEquals(1_110d, WorkloadMath.peakFromFactor(average, 4d), 15d);
        assertEquals(1_000d, WorkloadMath.peakHourQps(heartbeats, 0.15d), 1d);

        double watchSeconds = dau * 1_800d;
        double concurrent = WorkloadMath.averageConcurrency(watchSeconds);
        assertEquals(4_170d, concurrent, 5d);
        double peakViewers = WorkloadMath.peakFromFactor(concurrent, 4d);
        assertEquals(42_000d, WorkloadMath.megabitsPerSecond(peakViewers, 2.5d), 500d);
    }

    @Test
    void progressUpsertDoesNotImplyAShard() {
        double dau = 200_000d;
        double newPairs = dau * 3d * 0.20d;
        double bytesPerDay = newPairs * 256d;
        double year = bytesPerDay * 365d * 2d; // replica copy we operate
        assertEquals(2.3e10, year, 2e9); // about 23 GB, order check
    }
}

The tolerances are wide on purpose. An envelope that needs a one-percent assertion has become a forecast. The year-stock assertion checks order of magnitude: 120,000 × 256 × 365 × 2 = 2.24 × 10^10 bytes, about 22 GB. The test's delta is 2 GB. A refactor that drops the replica factor fails. A refactor that "simplifies" heartbeats into registered-user concurrency fails the viewer assertion by orders of magnitude, which is the failure we want.

Decision record

ADR-103: Classroom capacity envelope and the path video bytes may take

• Status: Accepted
• Context: The classroom serves lesson video, progress, and catalog reads. A draft design proxied video through the API and proposed a sharded event store for heartbeats.
• Drivers: the watch-day envelope and the launch-day envelope above. All inputs are assumptions until analytics replaces the table.
• Options: (A) One QPS for "the platform," video through the application, heartbeats as OLTP events. (B) Separate envelopes; video from object storage via the edge; OLTP holds latest position only; raw heartbeats, if kept, are a retained object-storage flow.
• Decision: Option B. Design peaks: about 1,100 heartbeat QPS and 16,700 concurrent viewers at 2.5 Mb/s, about 42 Gb/s at the edge. Launch catalog reads are a separate 2,700 QPS burst. Headroom policy is 2× on those peaks for benchmark targets. Object storage is not multiplied by three.
• Consequences: The API must not grow a byte-range proxy "just for this one client." Progress history in the primary reopens the record. A measured DAU ratio above 25 percent, or a bitrate above 6 Mb/s, reopens the edge budget. The launch cache, when it exists, is sized for the launch envelope.
• Reopen when: any row in the sensitivity table's "breaks when" column becomes true, or a benchmark at 2× heartbeat peak misses its latency budget. Missing the benchmark is not permission to skip the benchmark.

Validation

Replace one assumption at a time and confirm the conclusion column. Run the test. Have a second person recompute peak egress on paper: concurrency, times bitrate, units in megabits. The unit check is the entire review for that line. Then schedule the OLTP benchmark; the envelope does not claim it has been run.

Expected result

A reader who disagrees with the DAU ratio can change one row and see whether the shard comes back. A reader who wants video through the API has to explain 42 Gb/s of application egress. A reader who wants every heartbeat in PostgreSQL has to explain 2 TB a year of events and the retention job they are declining to run. The architecture is the set of those refusals.

Enterprise Real-World Scenarios

The terabit sketch

The platform review opened with a slide titled "2 million learners, HD video." The multiplication was registered users times 5 Mb/s. Someone had already asked a network vendor about a multi-terabit commitment and a design partner. The classroom squad spent a week on a custom caching tier to match a number nobody would ever observe.

Rebuilding the funnel took an afternoon. Registrations are 2 million. DAU at the ratio support could defend from the last term was closer to 8 percent than to 100 percent. Mean watch among those DAU was under half an hour, because most sessions are one lesson, not a day of playback. Concurrency fell from millions to the tens of thousands at the peak. Egress fell from terabits to tens of gigabits. That is still a serious CDN and object-storage problem. It is not a custom global network.

The decision record replaced the vendor thread. The sensitivity line was blunt: even if the DAU ratio is wrong by 5×, bytes still belong at the edge, and they still do not belong in a bespoke backbone. The sketch that survived was the boring one: object storage, an edge cache, the application limited to metadata and heartbeats. Finance asked why the first number had been so large. The answer was a unit error in the population, not in the arithmetic. Registered users had been treated as concurrent viewers. The postmortem action was to require the funnel in every capacity review, with the intermediate concurrency written down.

A quieter defect came with the correction. The revised model stored every heartbeat in the primary because "the QPS is only a thousand." At 6 GB of new rows a day the disk was fine for a month and unhappy after a year of undeleted events, and the autovacuum work competed with the upserts. The envelope's second conclusion — latest position in OLTP, raw events elsewhere, with retention — came out of that incident, not out of the first slide. Both failures are estimation failures. One is a population error. The other is a retention error. They produce different wrong architectures.

Launch morning versus Thursday night

Academies launched a mandatory compliance course to 400,000 learners with a Tuesday 09:00 start. The video edge, sized from the watch envelope, was quiet for the first hour. The catalog API, sized from Thursday's 140 reads per second, saturated. Timeouts retried. The launch window's true read rate matched the burst model in this lesson, which nobody had written down because "we already did capacity for the classroom."

The repair for the morning was a cache of course outlines, which lesson 27 designs, populated before 09:00. The repair for the process was a second envelope with a window fraction. Thursday's peak factor is the wrong tool when 40 percent of a day's browsing is scheduled into two hours by a customer. The progress database was almost idle during the incident, which confused the first responders into scaling the database. The saturated element was catalog read QPS. Scaling the bored element is what you do when the model has only one number labeled "load."

The ADR gained a line: a calendar launch uses the launch envelope, and a green video dashboard does not clear it. The next launch was rehearsed at the burst rate against the catalog, with the seat-count write still hitting the primary. That rehearsal is a validation step the arithmetic cannot perform for you. The arithmetic's job was to name the rate to rehearse.

Best Practices

• Start from DAU and actions, never from registered users times a bitrate.
• Show 86,400 in the denominator. A reviewer should recompute your average in one line.
• Cross-check a peak factor with an hour share. When they diverge by 10×, you have two shapes, not a mess to average.
• Give launches a window fraction. A daily peak factor hides a 09:00 start.
• Keep video, OLTP writes, and catalog reads in separate lines. Report the resource each one stresses.
• Use Little's law for mean concurrency, and state that the window is stable. A retry storm is not a residence time.
• Stay in bits until egress is computed, then convert once.
• Separate flow from stock. Retention produces a steady stock only when deletion actually happens.
• Multiply disk by copies you operate. Do not multiply object storage by a folklore replication factor the provider already includes.
• Put overhead in the row-size assumption once. Do not apply it again as a trailing ×3.
• Write the sensitivity: which input, moved by how much, changes the architecture.
• Record headroom as policy on a finished model. Benchmark the one rate the envelope cannot prove, such as upsert latency at 2× peak.
• Keep the model in code with order-of-magnitude tests so slides cannot drift.

Common Mistakes and Anti-Patterns

Registered users as concurrency. The population that has an account is multiplied by the bitrate of a stream. The number is enormous and the architecture becomes a fantasy network. It happens because the registration number is the only number in the brief. The repair is the funnel: ratio, then watch time, then concurrency, then bits.

One QPS for the product. Heartbeats, catalog reads, and video bytes are added, or only one of them is computed. The team shards a quiet primary and forgets the edge, or caches video metadata and still proxies bytes. The repair is one line per resource, with the envelope that drives it.

Peak factor on a launch. A factor measured on a normal Thursday is applied to a morning when a customer starts a cohort together. The average looks safe and 09:00 falls over. The repair is a window fraction for any load a calendar can synchronize.

Events kept forever in the serving store. Write QPS looks survivable, so every heartbeat is inserted into the primary "for analytics." A year later the stock, the indexes, and vacuum dominate. The repair is the retention formula and a different store for the stream, decided while the daily flow is still a small number of gigabytes.

Double-counted replication. Application disks are multiplied by three, and object storage is multiplied by three again, because a blog post about Hadoop said so. The bill and the design both inflate. The repair is a column called "who operates the copy."

Bits and bytes. A 2.5 MB/s assumption sneaks in where 2.5 Mb/s was meant, or the reverse. Egress is wrong by eight. The repair is to write the unit on every figure and to convert in one labeled step.

False precision. A spreadsheet prints 1,111.111 QPS and a cluster size of 17.4 instances. Nobody can see which assumption dominates, and the instance count implies a benchmark that never happened. The repair is two significant figures, a sensitivity table, and an explicit "benchmark before you claim cores."

Troubleshooting and Design Review

Symptom: The design's hero number is multi-terabit, and the DAU ratio is missing.

Likely cause: registered users were used as concurrent viewers.

Investigation: ask for concurrency as a separate line. Divide daily watch-seconds by 86,400. Compare.

Corrective action: rebuild the funnel before discussing vendors. Keep the scary number in the appendix as a rejected model so it does not return as folklore.

Symptom: The primary ran out of disk although write QPS matched the envelope.

Likely cause: the envelope sized the rate and forgot the stock, or a retention job never shipped, or every event landed in OLTP.

Investigation: compare bytes per day on the volume with flow × days since launch. Check whether the table is positions or events.

Corrective action: move the event stream, enforce retention, and put stock in the envelope beside QPS. Scaling the instance class does not fix an unbounded table.

Symptom: Video dashboards were green during an outage of the catalog.

Likely cause: one envelope. The launch stressed reads. The watch envelope was the only one reviewed.

Investigation: plot catalog QPS against the launch window fraction. Ignore the edge bitrate for this incident.

Corrective action: add the launch envelope and rehearse it. Page the people who own the saturated tier, not the tier that looks important.

Symptom: Origin egress is an order of magnitude above 5 percent of edge egress.

Likely cause: the miss-ratio assumption is false. Cache keys may include a query string that changes every player session, or TTLs are shorter than the lesson.

Investigation: miss ratio by URL shape, not by a single global average. A few URLs can dominate.

Corrective action: fix the cache key and the TTL. Do not buy origin capacity to honor a broken key. Revisit the 5 percent row after the fix so the envelope matches production.

Symptom: Two engineers' envelopes differ by 8× and both "followed the doc."

Likely cause: a bits/bytes slip, a missing replica, or a peak factor applied twice (once in the hour share and again as a multiplier).

Investigation: recompute from the assumption table with the shared class. Look for a second silent factor.

Corrective action: one implementation of the formula, tests with wide tolerances, units in the names. Delete the spreadsheet that reimplements it.

Design-review questions: Where is the DAU ratio? Can I recompute average QPS from daily events in one division? Do the peak-factor and hour-share models agree, and is there a launch window they both miss? What is peak concurrency, and which bitrate produced the gigabits? Which bytes are retained, for how long, and who operates the second copy? Which conclusion flips if one named input moves by 10×? What must be benchmarked because the envelope refused to invent a core count?

Interview Questions and Answers

Q1. How do you get from "two million users" to a request rate?

I refuse to multiply two million by anything until I have a DAU ratio and an action. Two million times 10 percent is 200,000 DAU. Times 120 heartbeats is 24 million events a day. Divided by 86,400 is about 280 QPS average. A peak factor of four, cross-checked against a busy hour that holds 15 percent of the day, lands near 1,000 to 1,100 QPS. I label every one of those inputs as an assumption. Registered users never become QPS directly.

Q2. Why do you cross-check a peak factor with an hour share?

They encode the same day in different ways, so agreement is evidence the shape is simple. A factor of four on a mean of 278 QPS is about 1,100. Fifteen percent of 24 million events in one hour is 1,000 QPS. When a launch puts 40 percent of browsing into two hours, both Thursday models are wrong, and I switch to fraction × daily / windowSeconds, then apply a burst inside the window if I have a reason. I do not average a launch into a year of Thursdays.

Q3. What does Little's law let you claim, and what does it not?

In a stable window, mean concurrency equals arrival rate times mean residence time. For video I often use the equivalent form: daily watch-seconds divided by 86,400. I do not claim a latency, a percentile, or a queue length during overload. A system accumulating retries is not stable, so the law is the wrong tool for the incident and the right tool for the average day. Latency under load is lesson 25.

Q4. Walk through the video bandwidth estimate and the unit check.

Average viewers are daily watch-seconds over 86,400, about 4,200 for this classroom. Times a peak factor of four is about 17,000 concurrent viewers. Times 2.5 megabits per second is about 42 gigabits per second. Divide by eight only when I need bytes, about 5 GB/s. If I had multiplied registrations by bitrate I would have reported terabits and designed a network for people who are asleep. I also state a miss ratio so origin egress is not silently equal to edge egress.

Q5. How do you estimate storage for data you delete and data you keep?

If deletion actually runs after R days and the write flow is steady, stock approaches flow times R. I quote the steady stock and I name the job that deletes. If the bytes stay, as published video stays, there is no steady state: I quote annual ingest and the horizon. For this classroom, heartbeat events would be gigabytes a day and need a retention rule; position upserts are tens of megabytes a day and about 22 GB a year including the replica. Video ingest is on the order of 20 TB a year and accumulates.

Q6. When do you multiply storage by three?

When I operate three copies. A primary and a replica is ×2 of disk I pay for and write to. Object storage that already includes provider durability is ×1 in the envelope, until I buy an extra region. Multiplying by three because a distributed filesystem used three replicas, and then multiplying again for the provider, is a double count. The record states who operates the copy.

Q7. Heartbeat QPS is only about a thousand. Why is that not permission to store every heartbeat in the primary?

Because QPS sizes the rate and retention sizes the stock. A thousand small inserts per second is on the order of 6 GB a day. Kept for a year, that is a couple of terabytes plus index and vacuum cost, on the same machine that must answer the latest-position read. I store the latest position in OLTP and, if analytics needs the stream, I put events in object storage with a retention window. The reopen condition is a product requirement that those events be queryable in the primary. That requirement has to pay for the stock.

Q8. How do you keep an estimate from becoming false precision?

I use two significant figures, I keep formulas in a small class with order-of-magnitude tests, and I publish a sensitivity table. I do not convert 1,100 QPS into a core count. I convert it into a benchmark target at the headroom policy, here 2×, and I say the benchmark has not been run. Precision that implies a measurement is a false architecture.

Q9. What is headroom, and what can it hide?

Headroom is a policy multiplier on a finished model, so a wrong-but-close DAU ratio does not force a redesign the week after launch. A factor of two is what this lesson uses. It hides a modestly wrong peak factor. It does not hide a missing workload, a bits-versus-bytes error, or unbounded retention. Those are structural. I never apply headroom twice, once in the model and once in the purchasing spreadsheet, without saying I did.

Q10. Two workloads share a product. How do you present them in an interview?

As separate envelopes with separate saturated resources. For the classroom I would say: watches produce about 1,100 heartbeat writes per second and about 42 Gb/s of edge video; launches produce a couple of thousand catalog reads per second for a short window; checkout writes are far smaller than either. I would then name the design consequence of each line. If the interviewer only hears one number, I have merged the model and I will probably scale the wrong tier.

Q11. The origin is hotter than your miss ratio. Is the estimate wrong?

The estimate's assumption is wrong, or the system is not implementing the path the estimate assumed. I treat a 5 percent miss ratio as a high-sensitivity input. I look at cache-key cardinality before I buy origin capacity. A query parameter that is unique per session produces a miss ratio near 100 percent, and no envelope survives that. After the key is fixed I replace the 5 percent row with the measured ratio. The method worked if the broken assumption was visible.

Q12. How does this lesson differ from a back-of-the-envelope primer?

The primers show the division by 86,400 and a storage product, which you still have to know. This lesson adds a second peak shape, Little's law as concurrency rather than as a slogan, retention versus accumulation, copies you operate versus copies a provider operates, a sensitivity table, and a decision record that says which architecture the numbers refuse. The worked classroom is three resources, not one famous QPS. The Java class exists so the next review changes an input instead of retyping a slide.

Hands-On Exercises

Exercise: Capacity envelope for a media-sharing service

Problem statement. TechLearningPro instructors will share short practice clips: learners upload a clip under 200 MB, and classmates watch it in the browser. Build a capacity envelope. This is not the classroom lesson-video system you just read; keep its bytes out of your tables except as a contrast. Use your own assumptions and label them.

Tasks.

1. Write the funnel: registered learners, DAU ratio, uploads per DAU, watches per clip, mean watch duration, bitrate or file size.
2. Compute average and peak upload QPS, watch QPS if watches are API calls, and concurrent plays. Cross-check the peak two ways. Give launch day its own window if a cohort watches the same clip at the start of class.
3. Compute daily ingest bytes and one-year stock. State retention and whether unpublish deletes the object. State who operates replication.
4. Compute edge egress and an origin egress under a miss ratio you declare. Show the bits-to-bytes step.
5. Say which resource saturates first and which architecture you refuse. Include "bytes through the application" as an option you must explicitly reject or accept.
6. Write five sensitivity rows: the input, the move, and whether the conclusion flips.
7. Note one benchmark the envelope cannot replace.
8. Write an ADR of half a page with a rejected option and a reopen condition.

Expected deliverables. Assumption table, arithmetic, a path diagram, the sensitivity table, and the ADR.

Evaluation rubric.

A terabit figure built from registrations, a ×3 on provider object storage with no second copy, or a single QPS that mixes uploads with video egress means the envelope is not ready for review.

Lesson Summary

A capacity envelope is a funnel plus a retention rule plus a peak shape. Registered users become DAU, DAU become actions, actions become a daily count, and 86,400 turns that count into an average rate. A peak factor and an hour share cross-check a normal day. A window fraction describes a launch. Little's law turns watch-seconds into concurrency, and concurrency times bitrate turns into egress, with the unit written down.

On the TechLearningPro classroom the three answers diverge on purpose. About 1,100 heartbeat writes per second and about 22 GB a year of position rows do not justify a sharded OLTP event store. About 42 Gb/s of video does justify an edge and forbids the application process from carrying the bytes. A launch can still demand thousands of catalog reads per second while the edge is idle. Headroom is a policy applied after those sentences exist. The decision record states the sentences and the inputs that would repeal them.

Lesson 25 takes the rate this lesson produced and asks what happens to latency as utilization climbs, and which element actually saturates when you add machines.

Further Reading

• Little, "A Proof for the Queuing Formula: L = λW" — the stable-system identity used here for concurrency. It is not a latency formula.
• Gregg, Systems Performance — how to confirm an envelope with utilization and saturation once you are allowed to measure.
• RFC 9110 and your CDN's public caching guide — the path that makes a miss ratio a real input. Lesson 26 uses the guides; this lesson only needs the ratio to be visible.
• PostgreSQL documentation on routine vacuuming and on max_connections — background for why a survivable write QPS can still be the wrong stock, without inventing an IOPS number.
• AWS Well-Architected, Reliability and Cost pillars — question banks that should be answered with an envelope, not with a larger instance class.
• Primers: /learn/high-level-design/capacity, /learn/system-design/back-of-envelope-estimation.
• This course: /learn/advanced-system-design/high-level-system-design-methodology, /learn/advanced-system-design/scalability-and-performance-fundamentals, /learn/advanced-system-design/architecture-performance-and-capacity-review.