Design a URL Shortening Service

Advanced System Design & Software Architecture · Lesson 49 · Real-World System Design Case Studies

Lesson Overview

TechLearningPro's spring campaign wanted a link a learner could type from a slide: https://go.techlearningpro.example/s/k7Qm2pL. Marketing also needed that same short name to land on a partner registration host the company had reviewed. Lesson 23 already designed lesson permalinks, and it refused this product on purpose. A permalink stores a course id and a lesson id and builds an https URI on our host. A shortener stores a destination the caller supplied. Those are different trust decisions. Mixing them would turn the classroom redirector into an open redirect.

This lesson is the long case for Link Desk, the shortener that lesson 23 deferred. The method is the one in high-level system design methodology: behavior, scenarios, labeled assumptions, estimates, APIs, data ownership, components, one scale path, failure modes, and a decision record. The primer at /learn/system-design/url-shortener and the high-level sketch at /learn/high-level-design/url-shortener name the usual boxes — a code, a cache, a 302. They do not decide whether a browser may cache the redirect forever, who is allowed to choose the destination host, what a retry of create must return, or what the redirector does when the database is slow. Those decisions change the schema and the status codes. This lesson writes them down.

Link Desk is a read-heavy, write-light service with a security problem that looks like a string problem. The string is a URI. The design allows https destinations whose host is on an explicit allowlist, allocates an unguessable code, redirects with 302 Found so a later disable can be observed, and records clicks off the redirect path. Capacity arithmetic, shown below from labeled assumptions, says a single relational primary plus a disposable cache meets the current peak. Sharding, a second region, and a public-shortener multiplier stay on a reopen list. The point of the case study is to show which number would pull each of them back, and which failure the user actually sees.

You will leave with a packet a staff engineer can reject: requirements, assumption table, API, schema, container diagram, failure table, and two records. Java and Spring appear where they make the contract executable — ResponseEntity, CacheControl, a transactional create with an idempotency key — without pretending a vendor's private shortener API is something we can call.

Learning Objectives

By the end of this lesson you will be able to:

1. Separate Link Desk from lesson permalinks and state which product is allowed to store a caller-supplied destination.
2. Write functional behavior, three quality scenarios, and an out-of-scope list that includes server-side fetches of the destination.
3. Estimate create rate, redirect rate, row storage, and click-log storage from labeled assumptions, and name the first saturated element.
4. Specify create, resolve, disable, and redirect contracts, including idempotency conflicts and the status codes 302, 404, 410, 409, and 503.
5. Choose a code-generation scheme whose collision behavior and guessability you can compute, and keep custom aliases in the same key space.
6. Place a cache on the redirect path with a staleness window that matches the disable scenario, using the rules from caching strategies.
7. Explain why click analytics must leave the redirect latency budget, and what accuracy you are willing to lose when the queue is down.
8. Review a shortener design by checking the allowlist, the redirect status, the idempotency key, and the scale-path sentence.

Prerequisites

• Lesson 23, the design sequence and the permalink decision this lesson must not undo. Permalink codes and Link Desk codes live in different tables and on different paths.
• Lesson 24, capacity estimation. Every figure below is recomputable from the assumption table. Peak factor is an assumption, not a measurement.
• Lesson 26, load balancing. Redirectors are stateless. Nothing in this design requires a session pinned to one JVM.
• Lesson 27, cache-aside, TTL as a staleness promise, and single-flight on a hot key. Link Desk uses those mechanisms. It does not re-teach them.
• Lesson 33, message queues, for the click stream. Lesson 35, rate limiting, for create and for code scanning. Lesson 37, idempotency, for the create retry contract.
• Comfort with HTTP status semantics in RFC 9110 and with java.net.URI. The allowlist check is application code. It is not a framework feature.
• The primers are vocabulary. If you have already read them, treat "base62 and Redis" as the starting sketch this packet has to justify or refuse.

Detailed Core Concepts

The product boundary

Link Desk has four actors. A marketer or instructor, authenticated by the company identity provider, creates a link. A learner or anyone else on the internet resolves a code and receives a redirect. A reviewer maintains the host allowlist. An analyst reads aggregated click counts that are allowed to lag.

In scope for this iteration:

• Create an https link to an allowlisted host, with an optional expiry and an optional custom alias.
• Resolve a code to 302, 404, or 410.
• Disable a link. Later resolves become 410 within the staleness window the cache is allowed to use.
• Emit a click event for aggregation. Counts are approximate under failure.
• Enforce per-caller and per-IP rate limits on create and on resolve.

Out of scope, written so a later diagram cannot quietly add them:

• Links whose host is any host on the internet.
• Server-side fetches of the destination to build a preview image. That fetch is server-side request forgery waiting for an allowlist mistake.
• Password-protected codes, A/B rotation of the destination, and in-place edits of the destination. Changing a target is disable plus create.
• Treating the code as a capability that bypasses enrollment or partner authorization. The destination page still authenticates the visitor.
• Exactly-once click counts, a second geographic region, and a public multi-tenant shortener for arbitrary customers. Each has a reopen condition later in the lesson.
• Lesson permalinks. Course and lesson identifiers stay in the classroom module from lesson 23.

The boundary is the architecture. A box that "also resolves permalinks" has merged two threat models.

Functional behavior

Creation succeeds only when the caller is authenticated, the idempotency key is new or repeats an identical request, the destination passes the URI policy below, and the code or alias is unused. The response body contains the code and the absolute short URL. It does not contain a secret beyond the code itself, because the code will be pasted into slides.

Resolution is unauthenticated. The service looks up the code. An active, unexpired row produces 302 with Location set to the stored destination. A missing code produces 404. A disabled or expired row produces 410 Gone, which tells a well-behaved client to stop retrying. Expiry is evaluated at read time against the stored expires_at. A nightly job may flip status to disabled for housekeeping; correctness does not depend on that job, because the read path checks the timestamp.

Disable is authenticated and authorized: the owner, or a trust-and-safety role. It sets disabled_at and deletes the cache entry. A repeated disable is a success. Disable does not delete the row. Analytics and abuse review still need to know the code existed.

Custom aliases are caller-chosen codes in the same primary key as generated codes. The alphabet and length rules are part of the API, not a second table that a redirector might forget to query.

Quality scenarios

Three scenarios are allowed to fail the design. More would be a catalog.

Redirect latency. A learner, on the public internet, requests an active code during the peak hour. The response is 302 with p99 latency under 80 ms when the mapping is in the edge cache of the service, and under 200 ms on a cache miss that reaches the primary, measured at the load balancer in the home region. The scenario does not promise that latency from another continent. A second region is how that promise would be made, and we have not made it.

Disable visibility. An owner disables a link. Within 60 seconds, at least 99 percent of subsequent resolves from a cold client observe 410 or a disabled result, measured at the load balancer. Clients that cached a 301 are outside this scenario because we will not send 301. The 60 seconds are the cache TTL. Writing "instant" next to a 60-second TTL is a failed design, which is the same rule lesson 27 applied to outlines.

Destination safety. An authenticated caller submits a destination whose scheme is not https, whose host is an IP address, which contains userinfo, or whose host is not on the allowlist. The create call returns 422 and stores nothing. A resolve never emits a Location header that failed this policy at write time. The scenario also covers a host that was allowlisted and later removed: new creates fail; existing rows are disabled by the review process, not left as a permanent exception.

Availability of resolve is a fourth constraint rather than a separate story: 99.9 percent of well-formed resolve attempts in a month receive 302, 404, or 410, excluding caller rate limits. 503 counts against that budget. We do not promise 99.99 percent. The arithmetic and the on-call load of a four-nines redirector are a different product.

URI policy

The destination is stored only after it parses as an absolute URI and passes every rule below. The rules exist because string prefix checks are how open redirects ship.

• Scheme is https. http, javascript, data, and empty schemes are rejected.
• Host is present. Userinfo (https://user:pass@host) is rejected.
• Host is a DNS name on the allowlist after IDNA conversion to ASCII (java.net.IDN.toASCII) and lower-casing. Comparison is exact. partner.example does not allow evil-partner.example or partner.example.attacker.test. A leading-dot suffix rule is a different product and is refused here.
• IP literals are rejected, including dotted decimal and IPv6. Allowlisting 127.0.0.1 by accident is a common preview-bot bug. We have no preview bot, and we still reject literals so a later bot cannot "just fetch it."
• Port is absent or 443.
• The URI is at most 2,048 characters. Fragments are stored if present and returned unchanged. Browsers do not send fragments to servers on the subsequent request; we do not use them for authorization.

The allowlist is data, owned by the review workflow, not a constant in a jar. Each row has a host, an approver, and a review_due date. A host past review is not eligible for new creates. Existing links are listed for the reviewer and disabled explicitly. Automatic disable on the due date is attractive and dangerous: a missed review would break a live campaign at midnight. The design makes the break a human action and makes the overdue host visible.

Code generation

A generated code is seven characters from a 62-symbol alphabet (0-9, A-Z, a-z). The space size is (62^7 = 3.52 \times 10^{12}). Entropy is (7 \times \log_2 62 \approx 41.7) bits.

The service draws a cryptographic random code and inserts it. The primary key rejects a collision, and the transaction retries with a new draw, bounded to three attempts, then fails the request with 503 so the client can retry the whole create with the same idempotency key. At the five-year row count computed later (about (1.8 \times 10^8) rows), the chance a single draw hits an existing code is about (1.8 \times 10^8 / 3.52 \times 10^{12} \approx 5 \times 10^{-5}), roughly one retry in twenty thousand creates. That is a unique-index problem, not a sharding problem.

A sequential counter encoded in base62 is shorter to explain and worse for this product. Codes would be enumerable. Campaign links are not bearer tokens for private content, so enumeration is not a full account takeover, but it does let a scanner walk every campaign, including the ones that were only meant for a single cohort chat. Random codes make scanning a rate-limit problem. Lesson 35's limiter on resolve, plus a short negative cache for 404, is the control. We do not pretend 42 bits is a long-term secret. Private lessons stay behind authentication at the destination, as lesson 23 required.

Hashing the destination, or hashing destination plus owner, looks idempotent and creates the wrong uniqueness. Two campaigns to the same URL need two codes so their click streams stay apart. A hash also leaks equality: anyone who can compute it learns that a code exists for that URL. Collisions of a truncated hash are indistinguishable from "we already issued this" unless a second column stores the full preimage and the read path checks it. The unique primary key on a random code is the smaller mechanism.

Custom aliases share the primary key. Rules: length 4 to 32, charset [A-Za-z0-9-], must not be a reserved word (health, api, admin, s), must not match the generated pattern exclusively in a way we cannot distinguish — generated codes are stored the same way, so uniqueness is the only collision rule that matters. A caller who wants spring gets it if it is free. A generated code that happens to equal an alias fails the insert and retries. One lookup path remains: primary key on code.

What is durable, what is disposable, what is approximate

Three consistency classes live in one product, and the API must not blur them.

The link row is strongly consistent on the primary. After a successful create, a resolve that reaches the primary sees the row. After a successful disable commits, a resolve that reaches the primary sees 410. Replicas, if we add them for read scaling, are not on the miss path until lag is inside the disable window. Lesson 29's replication lag is why v1 reads misses from the primary. At the miss rate computed later, the primary can afford it.

The cache is disposable and stale by agreement. It may serve an active mapping for up to 60 seconds after disable if the explicit delete failed. It must not be the only copy. A cache flush is an incident for latency, not an incident for durability.

Click totals are approximate. The redirect path does not update a counter in the link row. A counter on the hot row serializes every click of a campaign that is on a slide in a keynote, which is the hottest key we have. Events go to a log. Consumers aggregate. If the log is unavailable, the redirect still returns 302 and a metric records the drop. The product statement to marketing is: counts can miss events during a queue outage; they are not a billing source.

Where the primers stop

/learn/system-design/url-shortener is a fair interview sketch: read-heavy, base62, cache, async analytics. It does not have an allowlist, a disable window tied to TTL, an idempotency conflict, a negative cache, or a scale path that refuses to shard. /learn/high-level-design/url-shortener adds a requirements outline. This lesson is the packet you would take to design review at TechLearningPro, including the decision to keep Link Desk as a module until a measured rate says otherwise. Lesson 7's modular monolith is the default. A separate deployable is earned by connection count, release cadence, or a team boundary, not by the glamour of the interview diagram.

Architecture and Design Diagrams

The context diagram has one system inside the boundary. Marketers and anonymous resolvers are people. The identity provider, the click pipeline, and the allowlist review are explicit. The destination site is outside: we send the browser there, and we do not call it.

mermaidCopy
C4Context
    title Link Desk — system context
    Person(marketer, "Marketer or instructor", "Creates and disables campaign links")
    Person(visitor, "Visitor", "Opens a short link from a slide or chat")
    Person(reviewer, "Trust reviewer", "Approves destination hosts")
    System(linkdesk, "Link Desk", "Issues codes, redirects, and emits click events")
    System_Ext(idp, "Company IdP", "OpenID Connect for create and disable")
    System_Ext(dest, "Allowlisted destination", "Partner or TechLearningPro page the browser requests next")
    System_Ext(analytics, "Click pipeline", "Aggregates approximate counts")
    Rel(marketer, linkdesk, "Creates and disables links", "HTTPS")
    Rel(visitor, linkdesk, "Resolves a code", "HTTPS")
    Rel(reviewer, linkdesk, "Maintains the host allowlist", "HTTPS")
    Rel(linkdesk, idp, "Validates access tokens", "OIDC")
    Rel(visitor, dest, "Follows Location", "HTTPS")
    Rel(linkdesk, analytics, "Emits click events", "queue")

Containers inside Link Desk stay few. The redirector and the write API are one deployable until the scale sentence changes. The cache and the queue are separate processes because their failure modes differ: a dead cache must degrade to the database, and a dead queue must not fail the redirect.

mermaidCopy
flowchart LR
    visitor[Visitor browser]
    owner[Authenticated owner]
    edge[TLS load balancer]
    app[Link Desk application]
    idp[Identity provider]
    db[(Link and allowlist tables)]
    cache[(Code cache)]
    queue[Click queue]
    agg[Aggregator]

    visitor --> edge
    owner --> edge
    edge --> app
    app --> idp
    app --> cache
    app --> db
    app --> queue
    queue --> agg

Resolve is the hot path. The application answers from cache when it can, and it never calls the destination host.

mermaidCopy
sequenceDiagram
    participant B as Browser
    participant L as Load balancer
    participant A as Link Desk
    participant C as Code cache
    participant D as Primary database

    B->>L: GET /s/{code}
    L->>A: forward
    A->>C: get code
    alt cache hit active
        A-->>B: 302 Location destination
    else cache hit disabled or expired
        A-->>B: 410 Gone
    else miss
        A->>D: select by code
        alt row active and unexpired
            A->>C: put active mapping TTL 60s
            A-->>B: 302 Location destination
        else row disabled or expired
            A->>C: put disabled TTL 60s
            A-->>B: 410 Gone
        else no row
            A->>C: put negative TTL 10s
            A-->>B: 404 Not Found
        end
    end
    A-->>A: emit click only after 302 decision, async

Create is the trust path. The allowlist check happens before any code is visible to a resolver.

mermaidCopy
sequenceDiagram
    participant M as Owner client
    participant A as Link Desk
    participant I as Identity provider
    participant D as Primary database

    M->>A: POST /v1/links Idempotency-Key
    A->>I: validate token
    I-->>A: subject
    A->>A: parse URI and match allowlist
    alt policy reject
        A-->>M: 422 destination rejected
    else idempotency key replay same hash
        A-->>M: 200 existing code
    else idempotency key replay different hash
        A-->>M: 409 conflict
    else new
        A->>D: insert link and idempotency row
        D-->>A: committed code
        A-->>M: 201 code and short URL
    end

Technical Explanation

Read path and the status code

Resolve is safe and idempotent. Any replica of the application can serve it. The load balancer uses a plain round-robin or least-connections policy across healthy instances. Health is "process up and able to talk to the database or serve cache," with a nuance: if the database is down and the cache is up, the instance stays in service and only cache hits succeed. Misses return 503. Pulling the instance out because the database is down would throw away the cache hits that keep a keynote slide working. Lesson 26's health check has to encode that degraded mode, or the load balancer will "help" by draining every instance together.

302 Found is the success status. RFC 9110 allows the method of the follow-up request to change to GET for historical clients, which is what we want for a link opened in a browser. 301 Moved Permanently is the wrong contract: browsers and some intermediaries may cache it indefinitely, and a disable would not reach them. 307 and 308 preserve the method and body. A short link is almost always a GET from a human. Preserving a stray POST to the short URL would forward that body to a partner. We refuse that. Cache-Control: no-store on the 302 asks shared caches not to retain the redirect. Our own code cache is a separate system with its own TTL. Mixing those two caches — "the CDN cached the 301" — is how disable stops working while dashboards stay green.

The Location value is the destination stored at create time, not a value recomputed from untrusted query parameters. Resolve accepts only the code. There is no ?url= parameter. An API that redirects to a query parameter is an open redirect even if the rest of the design is careful.

Write path and idempotency

Create carries Idempotency-Key, a client-generated token scoped to the owner. Lesson 37's rule applies directly: the same key and the same request hash return the original code; the same key and a different hash return 409 Conflict; a lost response after commit must not allocate a second code. The implementation stores the key and a hash of the canonical request in the same transaction as the link row. A unique constraint on (owner_id, idempotency_key) is the lock. Two racing retries: one insert commits, the other sees the unique violation, reads the stored hash, and either returns the code or returns 409.

Disable is idempotent without a client key. The statement sets disabled_at if it is null and always deletes the cache key after commit. Deleting the cache key before commit is the bug lesson 27 names: a resolver can refill the cache from the still-active row, and then the commit lands, leaving a fresh active entry. Order is commit, then delete. A crash between them leaves a stale active entry until TTL. That is the 60-second scenario, not a bug to paper over with a synchronous distributed transaction.

Cache, hot keys, and negative caching

Cache-aside, as lesson 27 defines it: the application reads the cache, and on miss reads the primary and fills the cache. Values are small — code, status, destination, expiry — on the order of two kilobytes. At a 95 percent hit ratio, which is an assumption until measured (A8 below), peak database reads are about 5 percent of peak redirects.

A single campaign code can dominate. The assumption A6 says 20 percent of codes take 80 percent of clicks, and a launch day can be harsher: one code, most of the hour. That key is hot inside a shared cache as well. The containment from lesson 27 is a process-local cache of a few thousand codes with a TTL inside the same 60-second window, plus single-flight so a miss stampede collapses to one primary read per application instance. Ten instances missing together are ten queries, which the primary can absorb at this size. A lock service around one key would be a bigger outage mode than the stampede.

Unknown codes get a negative entry for 10 seconds. Without it, a scanner drawing random seven-character strings turns every guess into a primary read. With it, a repeated guess is cheap, and a code created during those 10 seconds can resolve as 404 until the negative entry expires. Creates are rare and owners can tolerate a few seconds before a brand-new link works from a cache that just cached the miss. The create path deletes the negative key for that code after commit, so the owner who immediately clicks is not waiting. A scanner who guessed the code in the same 10 seconds is the residual. Rate limiting on resolve is the second control: anonymous clients are limited per IP, authenticated owners have a higher ceiling, and excess receives 429.

Scale path

Under ten times the peak redirect rate in the assumption table, the primary's miss-path reads saturate before application CPU, and only if the hit ratio falls. The tactic already in the design is the cache. The next tactic, if the hit ratio is high and the primary is still hot, is a read replica for misses, with lag bounded by the disable window. Sharding by code hash is the tactic after storage or write rate exceeds one primary. Nothing in today's numbers selects it. Writing "shard from day one" spends lesson 30's operational cost — rebalancing, scatter queries for an owner's link list, cross-shard idempotency — on a problem we have not measured.

A second region reopens when the latency scenario grows a promise for another continent, or when availability must survive a region loss. Active-active writes on one code space need a conflict rule for aliases. The simple multi-region step is read replicas plus regional caches, with creates still in the home region. Creates are a few per second. Cross-region create latency is acceptable. Cross-region resolve latency is the thing a local cache fixes.

Security

The identity provider authenticates create, disable, and allowlist changes. Resolve does not call the identity provider. A dependency on the IdP for every redirect would make a login outage a link outage, and the redirect has no user-specific decision to make. Authorization at the destination remains the destination's job.

Threats the design names:

• Open redirect and phishing. Mitigated by the URI policy and the absence of a pass-through query parameter. A code is a pointer to a reviewed host, not a general browser steering wheel.
• Allowlist confusion. Exact host match after IDNA. Review dates. No automatic fetch.
• Scanning and enumeration. Random codes, 429, negative cache, and no sequential ids in the public code.
• SSRF. No component retrieves the destination. A future unfurl service needs its own record, its own network egress policy, and a default of off.
• Privilege. Owners read and disable their rows. Trust-and-safety can disable any row. The owner list query is WHERE owner_id = subject. An id in the URL is not authority.
• Abuse of a compromised marketer account. Rate limits per subject, audit log of creates, and the ability to disable. This is not solved by a longer code.
• Log leakage. Destinations can contain query tokens if a caller puts them there. The API rejects known patterns such as access_token and code on the query string of the destination, and the click event stores the code, not the full destination. Callers who embed secrets in URLs are still wrong. The reject rule catches the obvious cases.

Transport is HTTPS at the load balancer. Codes in access logs are operational data. Retention of access logs follows company policy and is shorter than retention of the link row.

Failure handling

Timeouts are part of the latency scenario. An unbounded database call means the p99 budget is a wish. The application budget for a miss is 150 ms of dependency time inside the 200 ms scenario, leaving room for the network to the balancer.

Trade-offs worth recording

Random code versus database sequence. Random costs a rare unique retry and gives you scanning resistance. A sequence is gap-free and enumerable. Link Desk takes random. A gap-free public identifier is not a requirement.

302 plus our cache versus 301 plus browser cache. 301 removes load and also removes disable. We take the load. At the computed peak the load is small.

Synchronous counter versus async events. A counter is easy to display and hard to keep off the hot key. Events are operationally heavier and honest about loss. We take events because the hot key is the failure mode of a successful campaign, which is when marketing most wants the counter.

Module versus service. One deployable beside the classroom application keeps one on-call path. The reopen condition is a team split or a resolve peak that competes with classroom deploys. The record should name the number, not the mood.

Store the destination versus store a partner id and a path template. A template is safer and cannot express the campaign URLs marketing actually uses. We store the destination under the URI policy. If the allowlist cannot be operated, the alternative is to stop offering partner links, not to open the policy.

Practical Implementation or Design Walkthrough

Assumption table

Capacity arithmetic

Average create rate:

[ \frac{100{,}000}{86{,}400} \approx 1.2 \text{ creates/s} ]

Peak create rate uses A3:

[ 1.2 \times 5 \approx 6 \text{ creates/s} ]

Average resolve rate:

[ \frac{10{,}000{,}000}{86{,}400} \approx 116 \text{ resolves/s} ]

Peak resolve rate:

[ 116 \times 5 \approx 580 \text{ resolves/s} ]

Five-year rows:

[ 100{,}000 \times 365 \times 5 = 1.825 \times 10^8 \text{ rows} ]

Row storage:

[ 1.825 \times 10^8 \times 500 \text{ bytes} \approx 91 \text{ GB} ]

An owner index on (owner_id, created_at) at roughly 40 bytes per entry is about 7 GB. This fits one primary with a synchronous replica for durability. Backup and the replica's disk are the capacity plan, not a cluster.

Miss-path reads at peak, under A8:

[ 580 \times 0.05 \approx 29 \text{ reads/s} ]

Twenty-nine primary reads per second plus six writes per second is a quiet relational database. The saturated element at today's peak is not the disk. It is the tail latency of a miss if the pool is exhausted by something else, which is why the cache and the timeout exist. At ten times peak, miss traffic is about 290 reads per second if the hit ratio holds, still plausible for one primary. At ten times peak with a cold cache, 5,800 reads per second is the number that forces the replica or a warmer cache before anyone draws a shard map.

Click volume over 90 days:

[ 10{,}000{,}000 \times 90 = 9 \times 10^8 \text{ events} ]

[ 9 \times 10^8 \times 100 \text{ bytes} \approx 90 \text{ GB} ]

That is an analytical store, partitioned by day, not an update to link.click_count. Five-year raw clicks at this rate would be about 1.8 TB. We do not keep raw clicks that long. Daily aggregates survive; raw events expire.

Bandwidth is not the driver. A 302 response is under a kilobyte. Peak:

[ 580 \times 1{,}000 \text{ bytes} \approx 0.6 \text{ MB/s} ]

Anyone who sizes Link Desk like a video platform has imported the wrong case study. Lesson 51 is where bytes dominate.

Public-shortener multiplier, labeled M1, not a forecast: one hundred times the resolve peak is about 58,000 resolves per second. Hit ratio 95 percent leaves about 2,900 primary reads per second, which is where a replica set or a larger cache tier becomes the design, and where code-hash sharding starts to earn its keep if writes or storage have grown in proportion. M1 is how you talk about the interview-scale sketch without pretending TechLearningPro has that traffic.

API

Base URL for writes: https://go.techlearningpro.example. Short resolves use https://go.techlearningpro.example/s/{code} so the application paths /v1 and /health never collide with a code. Reserved first segments are rejected as aliases.

POST /v1/links

• Auth: bearer token from the company IdP.
• Header: Idempotency-Key required.
• Body fields: destination (string), expiresAt (optional RFC 3339), alias (optional).
• 201 body: code, shortUrl, expiresAt, status.
• 200 when the idempotency key replays the same hash.
• 409 when the key replays a different hash.
• 422 when the URI policy fails, with a machine-readable reason (scheme, host, port, userinfo, length, literal-ip, query-secret).
• 429 when the caller or IP is over its limit.
• 503 when the primary cannot commit.

GET /s/{code}

• No auth.
• 302 with Location and Cache-Control: no-store.
• 404 unknown.
• 410 disabled or expired.
• 429 rate limit.
• 503 miss and primary unavailable.

POST /v1/links/{code}/disable

• Auth, owner or trust role.
• 204 on success, including a repeat.
• 404 if the code does not exist or is not visible to the caller. Hidden existence for other owners' codes is 404, not 403, so we do not confirm the code.

GET /v1/links?owner=me

• Auth. Page of the caller's links. This is the query that wants the owner index. It is not the hot path.

GET /v1/allowlist and PUT /v1/allowlist/{host} are review tools, role-gated, audited. They are part of the safety scenario, not an afterthought.

Schema

sqlCopy
CREATE TABLE destination_host (
    host         varchar(253) PRIMARY KEY,
    approver_id  uuid NOT NULL,
    approved_at  timestamptz NOT NULL,
    review_due   date NOT NULL
);

CREATE TABLE link (
    code              varchar(32) PRIMARY KEY,
    owner_id          uuid NOT NULL,
    destination       varchar(2048) NOT NULL,
    destination_host  varchar(253) NOT NULL,
    status            varchar(16) NOT NULL,
    expires_at        timestamptz,
    created_at        timestamptz NOT NULL,
    disabled_at       timestamptz,
    idempotency_key   varchar(80) NOT NULL,
    request_hash      bytea NOT NULL,
    CONSTRAINT link_status_chk CHECK (status IN ('active', 'disabled')),
    CONSTRAINT link_owner_idem UNIQUE (owner_id, idempotency_key)
);

CREATE INDEX link_owner_created ON link (owner_id, created_at DESC);

destination_host is denormalized onto the link so a later removal from the allowlist does not require parsing URIs to find affected rows. Status stays active until disable; expiry is a predicate on expires_at. The check constraint keeps unknown statuses out. There is no click_count column.

Click events, when the queue consumer lands them, are a separate table or an object-storage log partitioned by day: code, resolved_at, edge_region, a coarse hash of the client address. No raw address if policy says so. No destination string.

Spring sketch

The redirect handler stays thin. Policy lives in the write service, which is the only place a destination enters the database.

javaCopy
@GetMapping("/s/{code}")
public ResponseEntity<Void> resolve(@PathVariable String code) {
    ResolvedLink link = links.resolve(code);
    if (link.outcome() == Outcome.MISSING) {
        return ResponseEntity.notFound().build();
    }
    if (link.outcome() == Outcome.GONE) {
        return ResponseEntity.status(HttpStatus.GONE).build();
    }
    if (link.outcome() == Outcome.UNAVAILABLE) {
        return ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE).build();
    }
    return ResponseEntity.status(HttpStatus.FOUND)
            .location(link.destination())
            .cacheControl(CacheControl.noStore())
            .build();
}

Create validation uses the platform URI type and IDNA, then an exact host lookup. URI.create does not apply IDNA by itself; the host must be converted before the allowlist comparison. Reject userinfo before you look at the host, because a URL of the form https://allowlisted.example@evil.test has surprised more than one hand-rolled parser. java.net.URI puts allowlisted.example in userinfo and evil.test in the host. Trust the host field after a real parse. Do not substring.

javaCopy
public URI requireAllowed(String raw) {
    URI uri = new URI(raw);
    if (!"https".equalsIgnoreCase(uri.getScheme())) {
        throw new DestinationRejectedException("scheme");
    }
    if (uri.getUserInfo() != null || uri.getHost() == null) {
        throw new DestinationRejectedException("host");
    }
    if (uri.getHost().indexOf(':') >= 0 || looksLikeIp(uri.getHost())) {
        throw new DestinationRejectedException("literal-ip");
    }
    if (uri.getPort() != -1 && uri.getPort() != 443) {
        throw new DestinationRejectedException("port");
    }
    String host = IDN.toASCII(uri.getHost()).toLowerCase(Locale.ROOT);
    if (!allowlist.isApproved(host)) {
        throw new DestinationRejectedException("host");
    }
    return uri;
}

The create transaction inserts the idempotency row and the link row together. A DataIntegrityViolationException on the idempotency unique key reloads and compares hashes. A violation on the code primary key retries the random draw. Both cases are ordinary Spring @Transactional boundaries on one database. They are not a saga. Lesson 32's sagas appear when more than one service must commit. Here there is one writer.

Cache access uses a short get timeout. On timeout or error, resolve treats the cache as a miss and reads the primary, unless the primary budget is already gone, in which case the outcome is UNAVAILABLE. Single-flight can be a per-code lock in the JVM with a timeout, the same shape as lesson 27's stampede control. The lock must expire if the holder dies, so it lives with a TTL if it is shared, and it is process-local if it is only collapsing one instance.

Test strategy

Tests that matter for this design, beyond a happy-path redirect:

• Each URI rejection reason, including userinfo confusion and a suffix lookalike host.
• Idempotency replay: same body, different body, and a simulated timeout after commit.
• Disable then resolve on the primary returns 410 even if a cache delete is forced to fail, once the TTL is set to zero in the test.
• Negative cache: an unknown code, then an insert, then a resolve that must observe the delete of the negative key.
• Primary timeout on a miss returns 503 and does not write a cache entry.
• Queue failure still yields 302.
• A generated code never contains a reserved path segment.
• Load test at 580 resolves per second with one hot key, asserting primary QPS stays near the miss estimate.

An end-to-end test that only asserts 302 to https://example.com without an allowlist is testing a different, weaker product.

Decision records

ADR-049-1. Partner destinations live in Link Desk, not in the permalink table. Options: add a URL column to permalinks; run a separate module with an allowlist. Decision: separate module and table. Consequence: two resolvers, two caches, and a clear place for marketing. Reopen if the company abandons partner links entirely, in which case Link Desk can be retired rather than merged.

ADR-049-2. Redirect with 302 and a 60-second positive cache TTL. Options: 301; 302 with no cache; 302 with a day-long TTL. Decision: 302, Cache-Control: no-store, application cache TTL 60 seconds aligned with the disable scenario. Consequence: we pay for resolve traffic the browser could have swallowed, and disable is bounded. Reopen if a measured peak threatens the primary and product accepts a longer disable window.

Enterprise Real-World Scenarios

Keynote code, one hot key

During the spring launch, one code is on the title slide for forty minutes. Assume, for this incident exercise, that half of the day's ten million resolves arrive in that hour on that single code: about 1,400 requests per second. The process-local cache serves almost all of them after the first miss per instance. The shared cache holds one key. The primary sees a handful of fills. The failure mode that actually happens is operational: someone "fixes" the hot key by setting a one-day TTL so the database stays idle, and the speaker's "we retired that offer" disable is invisible for the rest of the conference. The scenario from the requirements is the page that should fire — disable age versus TTL — not a CPU graph.

The click pipeline falls behind because every one of those resolves emits an event. Redirect latency stays inside budget only if publish is asynchronous and buffered, with a drop policy when the buffer fills. Marketing's dashboard lags by minutes. The honest banner on the dashboard is part of the design. A synchronous increment of click_count on that row would turn the keynote into a single-row write hotspot, which is the outage the cache was supposed to prevent.

Partner host that no longer belongs to the partner

A registration vendor is acquired. Their domain lapses and is registered by someone else. The allowlist still contains the host, and last year's campaign links still resolve. A 60-second cache TTL does not save you: the row is legitimately active. The safety scenario's control is the review date plus a disable action. TechLearningPro's reviewer sees review_due in the past, lists links for that host through destination_host, and disables them. New creates already fail once the host is marked ineligible. The residual risk is the gap between lapse and review. Shortening that gap is a process with a calendar, not a new database. Automatic disable at review_due remains refused because it couples a missed meeting to a production redirect change. If finance later decides a lapsed host is worse than a broken campaign, that is a new record with a new default.

A second variant of this incident is an allowlist entry inserted as partner.example while the create path checked endsWith("partner.example"). notpartner.example would have been stored. The exact-match rule and a test for the lookalike are the correction. Incident review that only "adds the bad host to a block list" leaves the matcher broken.

Best Practices

• Write the out-of-scope list before the container diagram. Partner fetches, permalinks, and in-place edits should be visible as refusals.
• Tie every cache TTL to a scenario that names the same number. Disable visibility and negative-cache delay are those scenarios.
• Put the unique constraint and the idempotency key in one transaction on one database. Do not allocate the code in a side service that can succeed when the link insert fails.
• Return 410 for "used to exist" and 404 for "never existed or not yours." Clients and caches treat them differently.
• Keep click writes off the resolve thread. Measure drop rate. Tell analysts the number is approximate.
• Compare hosts with a URI parser and IDNA, exact match, no IP literals.
• Size the design from the assumption table, and keep sharding on the reopen list until a measured rate crosses the number in the record.
• Log the code and the decision, not the full destination, when destinations may contain query tokens.
• Use no-store on the HTTP redirect so intermediaries are not a second cache you forgot to purge.
• Review allowlist rows on a calendar. The list is a security control, not a configuration convenience.

Common Mistakes and Anti-Patterns

Hashing the URL as the code. It collides distinct campaigns, it is predictable, and truncation forces a read-time compare that the team will eventually skip. A random code and a unique primary key separate identity from the destination.

301 for a lower latency score. The browser cache then defeats disable. Latency looks better in a benchmark that never revokes anything. Production campaigns do get revoked.

A click counter on the link row. The most successful link becomes the hottest row. The outage correlates with the marketing win, which is an unpleasant way to discover a write hotspot.

Suffix allowlists and hand-rolled URL splits. endsWith and splitting on @ mis-parse userinfo and lookalike domains. Use a real URI parser and an exact host set.

Fetching the destination from the shortener "just to check it is up." That is SSRF, and it puts partner latency on the create path. Create stores a URI. It does not browse.

Negative caching without deleting the key on create. Owners click their new link, get 404 for the whole TTL, and file a bug. The scanner was the threat. The owner is the user. Delete the negative entry on commit.

Sharding the first sketch because the interview answer had shards. At six writes per second you will debug cross-shard idempotency before you ever fill a disk. Lesson 30 is the tool for when the estimate changes, not a default decoration.

One resolver for permalinks and campaign links. The permalink ADR forbade stored URLs. Folding Link Desk back in reopens that hole with no allowlist.

Authentication on the redirect path. It couples resolve availability to the IdP and does not authorize the destination. Private content stays at the destination.

Troubleshooting and Design Review

Symptom: Disable returns success, and browsers still land on the old offer an hour later.

Likely cause: the response was 301, or an intermediary cached 302 because Cache-Control was missing, or the application cache TTL was lengthened during an incident.

Investigation: capture the redirect status and response headers from a cold client. Compare with the cache TTL configuration and the row's disabled_at.

Corrective action: restore 302 and no-store. Set the application TTL back inside the scenario. Purge the intermediary if one exists. Do not tell users to "hard refresh" as the design.

Symptom: Primary CPU climbs during a launch even though the hit ratio dashboard says 99 percent.

Likely cause: the hit ratio is averaged across all codes, while one hot key is evicting and missing; or negative-cache misses from a scanner are counted as hits incorrectly; or click-count updates are hitting the primary outside the cache.

Investigation: break out QPS and hit ratio by code. Inspect write QPS on link. Sample primary statements.

Corrective action: local cache and single-flight for the hot code. Remove any synchronous counter. Rate-limit the scanner. Read lesson 27's hot-key section before adding shards.

Symptom: Create returns 201 twice for one user action, two different codes.

Likely cause: the client omitted Idempotency-Key, or the server recorded the key after the link insert in a second transaction that sometimes did not run.

Investigation: logs for the two codes and the key. Transaction boundaries in the create method.

Corrective action: require the key, insert both rows in one transaction, and on unique violation return the original. Add the timeout-after-commit test.

Symptom: A link was created to a host the reviewer does not recognize.

Likely cause: matcher bug, or an allowlist write path that bypasses review, or IDNA confusion (xn-- lookalikes).

Investigation: stored destination_host, the raw destination, the allowlist row's approver and time.

Corrective action: disable the link. Fix the matcher. Audit allowlist writes. Add the lookalike test that would have failed the review.

Symptom: Resolves return 503 while the cache process is healthy and full.

Likely cause: health checks require the primary, so the load balancer drained every instance; or the application treats a cache hit as unusable when the primary probe fails.

Investigation: balancer target health, application logs for "served from cache," primary error rate.

Corrective action: serve hits in degraded mode. Page on miss 503 rate, which is the real user pain.

Design-review questions: What is out of scope? Which host rule rejects userinfo and suffix tricks? What status does disable produce, and which TTL matches it? What happens to a retried create? What is peak resolve QPS from the assumptions, and what is peak primary QPS after the hit ratio? What does the visitor see if the queue is down, if the cache is down, and if the primary is down on a miss? Which record keeps permalinks out of this table?

Interview Questions and Answers

Q1. Why does this design redirect with 302 instead of 301?

301 permits clients and intermediaries to store the mapping permanently. A campaign disable would not reach those clients, so the disable scenario would be false. 302 with Cache-Control: no-store keeps the browser from becoming a cache we cannot purge. We still cache inside Link Desk for up to 60 seconds, which is the window the scenario allows. I would use 301 only for a code that is contractually immutable, and I would say so as a different product. 307 is a poor fit because it can forward a non-GET method and body to a partner host.

Q2. How do you generate codes, and how do you know collisions are acceptable?

I draw a cryptographic random seven-character base62 string and rely on the primary key. The space is about (3.5 \times 10^{12}). At about (1.8 \times 10^8) rows the chance one insert collides is about (5 \times 10^{-5}), so the transaction retries a few times at most. I do not use a hash of the URL, because two campaigns to the same destination need two codes, and a hash is guessable. I do not use a public sequence, because it makes every code enumerable. Forty-two bits is enough to push scanning into the rate limiter. It is not enough to treat the code as a password for private content, so the destination still authenticates.

Q3. What is the scale path in one sentence, using your numbers?

At a peak of about 580 resolves per second and a 95 percent hit ratio, the primary sees about 30 reads per second, so the first sketch is one primary, a replica for durability, and a cache. Under ten times that peak the miss path is still the element to watch; the next tactic is a read replica for misses or a warmer local cache, and sharding waits until storage or write rate actually threatens one node. I can defend that from the assumption table. I cannot defend a shard map drawn before the division by 86,400.

Q4. How should a retried create behave?

The client sends Idempotency-Key. The server stores the key and a hash of the body in the same transaction as the link. The same key and hash return the same code. The same key and a different hash return 409. If the client times out after the commit and retries, it gets the original code rather than a second campaign. A unique constraint is the concurrency control. This is one database, so it does not need a saga or an exactly-once queue.

Q5. Where do you put the cache, and what may it get wrong?

Cache-aside on the code, TTL 60 seconds for active and disabled mappings, 10 seconds for misses. It may serve a disabled link as active for up to 60 seconds if the delete after commit failed, and that bound is the disable scenario. It may briefly 404 a brand-new code if a negative entry exists, so create deletes that key. If the cache process is down, resolve reads the primary. If the primary is down, hits still answer and misses return 503. I do not fill the cache when the primary read fails.

Q6. How do you stop the service from becoming an open redirect?

Resolve takes only a code. Create accepts a destination only after a real URI parse: https, no userinfo, no IP literal, port 443 or default, host exact-matched to an allowlist after IDNA. I do not use endsWith on the host. I do not fetch the URL from the server. Existing links to a host that leaves the allowlist are disabled by review. Lesson permalinks are a different resolver that does not store URLs at all.

Q7. Why is analytics asynchronous, and what do you tell the business?

A counter updated on the link row makes the most popular code the hottest write key, which is the launch-day failure mode. The redirect emits a small event after it has decided the status, and it does not wait for durability of that event. If the queue is down, the visitor still gets 302 and we count the drop. Analysts get a dashboard that can lag and can under-count during an outage. Those counts are not invoices. If finance needed exact clicks, that would be a different pipeline with a different latency budget, and I would not put it on the redirect thread either. I would stage it.

Q8. How do you handle a scanner guessing codes?

Random codes make the hit rate of guesses tiny. 429 per IP limits the attempt rate. A short negative cache keeps repeated misses off the primary. Create deletes the negative key so owners are not stuck. I watch primary read QPS and 404 rate together. If they climb in lockstep, the limiter or the negative cache is misconfigured. I do not lengthen codes as the first response unless the entropy calculation says scanning is actually feasible at the allowed rate.

Q9. When would you shard, and what is the shard key?

I shard when one primary cannot hold the rows or the write rate, which this estimate does not show. The key would be the code, because resolve is a point read and the code is the primary key. The owner's list API then needs a second lookup or a fan-out, which is the cost I am refusing to pay early. Idempotency keys are scoped to an owner; if links are sharded by code, the idempotency row has to live where the transaction lives, so I would shard both by owner or keep idempotency in a dedicated store. That design work waits for the reopen condition. Lesson 30 is the method when we get there.

Q10. What is the difference between this system and the lesson permalink design?

Permalinks store course and lesson ids and build a URI on our host, so they cannot send a browser to a partner. Link Desk stores a reviewed destination because marketing links require it, behind an allowlist, a separate table, and a separate resolver. Both use 302 or an equivalent non-permanent redirect, both treat cache TTL as a revoke window, and neither treats the code as a grant of access to private content. Merging the tables would undo the permalink decision.

Q11. The cache is stale and the primary is down. What do you serve?

If I have an active cache entry, I cannot tell whether a disable landed after the fill. The strict reading of the safety scenario is to avoid sending people to a destination I cannot confirm. The availability scenario wants the keynote link to keep working. I write this down rather than hiding it: for entries filled recently, degraded mode serves the cached 302 or 410 and pages a human; I do not invent a destination; I do not fall back to a default URL. If product cannot accept a stale 302 during a primary outage, the cache must not be used for success either, and the availability number has to be renegotiated. The failure table's current choice is to serve hits and 503 on misses, and to keep TTL short so the stale window is the one already in the disable scenario.

Q12. How would you change the design for a hundred times the traffic?

I would recompute: about 58,000 resolves per second at peak, about 2,900 primary misses per second at a 95 percent hit ratio, and about 9 TB of link rows over five years if creates scale too. I would add regional caches, read replicas for misses, and only then code-based sharding if a single primary's storage or write rate is actually exceeded. Creates can stay home-region longer than resolves. I would not start with a multi-master alias space. I would also remeasure the hit ratio and the hot-key share before buying shards, because a local cache might still be the cheaper move. The allowlist, the idempotency transaction, and 302 stay. Scale does not relax the open-redirect rules.

Hands-On Exercises

Exercise: Link Desk packet for a second brand

Problem statement. A second TechLearningPro brand, aimed at enterprise buyers, needs short links on a different host name. Destinations must stay on a stricter allowlist than the consumer brand. Some links expire in one hour because they are attached to a sales quote. Produce a design packet. Use your own volumes if you prefer, and label every assumption. Do not put these links in the lesson permalink table.

Tasks.

1. Write in-scope behavior and an out-of-scope list. Include whether sales may embed one-time tokens in the destination, and what you reject.
2. Write three scenarios: redirect latency, disable or expiry visibility, and destination safety. Use a numeric measure and a window.
3. Build an assumption table and compute peak create QPS, peak resolve QPS, five-year storage, and analytical storage for clicks. Show the division by 86,400.
4. Specify status codes for create, replay, conflict, policy rejection, resolve hit, unknown code, expired code, and primary failure.
5. Choose a code scheme and compute the collision probability at your row count. State how custom aliases share the key space.
6. Draw a context diagram and a resolve sequence. Justify any component beyond the application, the primary, the cache, and the click queue.
7. Write the scale-path sentence. Write failure notes for primary, cache, queue, and IdP.
8. Write one ADR whose rejected option is "suffix match on the partner domain," with a reopen condition.

Expected deliverables. Assumption table with arithmetic, API and schema, two diagrams, failure notes, and the ADR.

Evaluation rubric.

A packet that stores arbitrary URLs, uses 301 while promising instant disable, or shards at a few writes per second is not ready for review.

Lesson Summary

Link Desk is a shortener whose hard problem is the destination, not the QPS. Authenticated creates store an https URI only when the host matches an allowlist exactly. Resolves are unauthenticated 302 responses with Cache-Control: no-store, backed by a cache whose TTL is the disable window. Missing codes are 404, disabled and expired codes are 410, and a primary miss during an outage is 503. Codes are random, collisions are retried against a unique key, and custom aliases share that key. Creates are idempotent in one transaction. Clicks are events, allowed to be dropped, never a counter on the hot row.

The estimate — about 6 creates per second and 580 resolves per second at peak, about 90 GB of rows over five years, about 30 primary reads per second after cache — supports one primary, a durability replica, and a disposable cache. Sharding and a second region stay on reopen conditions tied to numbers or to a latency promise we have not made. Lesson permalinks remain a different system that does not store URLs.

The primers show the boxes. The packet shows which boxes the numbers have not earned, and which failure the visitor sees when each dependency is down. Lesson 50 applies the same standard to a system where the hard problem is a live connection and an ordered log, not a redirect.

Further Reading

• RFC 9110, HTTP semantics — 301, 302, 307, 404, 409, 410, 422, and 503 as a contract.
• RFC 3986, URI syntax, and the IDNA specifications — why host comparison goes through a parser and IDN.toASCII.
• Lesson 23 in this course, high-level system design methodology — the sequence this packet follows, and the permalink decision it preserves.
• Lesson 24, capacity estimation; lesson 26, load balancing; lesson 27, caching; lesson 30, sharding, for the reopen path; lesson 33, queues; lesson 35, rate limiting; lesson 37, idempotency.
• C4 model — context before containers.
• AWS Well-Architected Framework — reliability and security questions answered from the failure table.
• Primers, as vocabulary only: /learn/system-design/url-shortener, /learn/high-level-design/url-shortener, /learn/system-design/mock-interview-url-shortener.
• Next case study: design a messaging system.