API Gateway

Introduction

An API gateway is the single entry point for client traffic — handling routing, authentication, rate limiting, request aggregation, and protocol translation before requests reach internal microservices. Amazon API Gateway fronts thousands of AWS service endpoints and internal retail APIs with centralized throttling and SigV4 auth.

Real production story

Amazon's retail mobile app originally called 14 internal microservices directly from the client — each with different auth headers, timeout behavior, and TLS cert rotation. A certificate expiry on the inventory internal ALB broke add-to-cart company-wide because the app had hardcoded 14 endpoint configs.

The mobile platform team deployed Amazon API Gateway (and internal equivalents) as the sole client-facing edge: one TLS surface, Cognito JWT validation at gateway, route mapping to VPC Link private integrations, and WAF rules blocking credential stuffing. Client bundle shrank — one base URL. Incident mean time to patch cert dropped from "find which of 14 endpoints" to one gateway domain.

Business problem

Business pressure: Amazon retail clients (mobile, web, Alexa) need stable public APIs while backend services refactor weekly — clients cannot track 200 internal hostnames.

• Revenue at risk: Mobile checkout failures from backend endpoint misconfiguration directly reduce conversion.
• Engineering velocity: Mobile releases blocked on backend URL changes — gateway abstracts routing.
• Compliance / trust: WAF, bot detection, and geo-blocking enforced once at edge — not per service.

Architecture overview

API gateway responsibilities: routing, authn/authz, rate limiting, SSL termination, request/response transformation, optional aggregation. Not business logic — domain rules stay in services.

• Definition: Reverse proxy + policy engine at system edge.
• When to adopt: Multiple microservices exposed to external clients.
• When to defer: Single monolith with one API — ALB + app middleware sufficient.
• Operability: Gateway metrics (4xx, 5xx, latency, throttle count) are client-experience SLOs.

Architecture motivation

Why architects care: Gateway centralizes cross-cutting edge concerns so microservices stay focused on domain logic — not TLS, JWT parsing, or CORS on every service.

• Force: Many clients, many services, need stable external contract.
• Constraint: Internal services must not be internet-routable — zero trust network.
• Outcome: Single public surface, policy-as-config, backend free to move.

Internal architecture

Amazon retail API gateway topology:

  Internet / Mobile App
         │
         ▼
  ┌──────────────────┐
  │   CloudFront     │  (CDN for static + edge cache)
  └────────┬─────────┘
           ▼
  ┌──────────────────┐
  │   AWS WAF        │  bot control, geo block, rate rules
  └────────┬─────────┘
           ▼
  ┌──────────────────┐
  │  API Gateway     │  Cognito JWT authorizer
  │  /cart/* → Cart  │  Usage plans + API keys (partners)
  │  /order/* → Order│  Request validation (OpenAPI)
  │  /catalog/* → …  │
  └────────┬─────────┘
           │ VPC Link (private)
     ┌─────┴─────┬─────────┐
     ▼           ▼         ▼
  Cart svc   Order svc  Catalog svc
  (internal  (no public (internal ALB
   ALB only)  IP)        only)

Data flow

Primary path: Mobile POST /cart/items with Bearer token → Gateway validates JWT via Cognito authorizer → checks usage plan quota → transforms path to internal /v2/cart/items → VPC Link forwards to Cart service ALB → response mapped to client schema.

• Write path: Gateway adds X-Request-Id, X-Principal-Id headers; Cart service trusts gateway mTLS, not client token.
• Read path: Optional edge cache for GET catalog — TTL per route; cache invalidation via CloudFront.
• Aggregation: Avoid heavy aggregation at gateway — use BFF for complex composition; gateway routes only.

System design diagram

Two diagrams show the API Gateway topology and the primary request/event path used in production at scale.

Production code example

AWS API Gateway OpenAPI + Cognito authorizer — Amazon retail pattern:

# serverless.yml excerpt — API Gateway routes
functions:
  cartIntegration:
    handler: src/handlers/cart.proxy
    events:
      - http:
          path: cart/items
          method: post
          authorizer:
            type: COGNITO_USER_POOLS
            authorizerId: !Ref CognitoAuthorizer
          request:
            schemas:
              application/json: ${file(models/add-to-cart.json)}
          throttling:
            burstLimit: 5000
            rateLimit: 2000

resources:
  Resources:
    ApiGatewayUsagePlan:
      Type: AWS::ApiGateway::UsagePlan
      Properties:
        Throttle: { BurstLimit: 10000, RateLimit: 5000 }
        Quota: { Limit: 1000000, Period: DAY }

# VPC Link integration — backend never public
CartServiceIntegration:
  Type: AWS::ApiGatewayV2::Integration
  Properties:
    IntegrationType: HTTP_PROXY
    ConnectionId: !Ref VpcLink
    IntegrationUri: !Sub arn:aws:elasticloadbalancing:.../cart-internal

Enterprise case study

Amazon retail mobile API consolidation (2018–2020): Single API Gateway reduced client-side endpoint config complexity and centralized WAF policy for credential stuffing attacks during Prime Day.

• Before: 14 direct service endpoints; cert rotation incidents; inconsistent auth.
• Decision: API Gateway + Cognito + WAF + VPC Link; OpenAPI-driven route config.
• After: Zero client releases for backend URL changes; 99.99% edge availability SLO met.

Trade-offs

• Centralization vs bottleneck: Gateway is SPOF for client traffic — multi-AZ, auto-scale, and health checks mandatory.
• Logic creep: Business rules in Lambda@Edge or gateway transforms — resist; becomes undeployable monolith at edge.
• Latency hop: Extra network leg — acceptable for policy; unacceptable for heavy aggregation.

Security considerations

Security is architectural: Gateway is zero-trust enforcement point — internal services accept only gateway identity.

• Identity: OAuth2/OIDC at gateway; internal mTLS from gateway to services.
• Data: WAF blocks injection; request size limits prevent payload attacks.
• Supply chain: OpenAPI spec as contract — gateway validates request shape before backend.

Scalability analysis

Scale dimensions: Amazon API Gateway scales automatically but usage plan limits and Lambda authorizer cold starts need tuning at Prime Day traffic.

• Horizontal scale: Gateway managed scale; backend services scale independently behind VPC Link.
• Hot spots: Single popular route (/cart) dominates — per-route throttle and backend pool isolation.
• Cost: Per-request gateway pricing — cache at CloudFront for read-heavy catalog routes.

Failure scenarios

What breaks: Authorizer Lambda timeout blocks all traffic; misconfigured route sends payment traffic to staging.

• Authorizer cascade: Cognito slow → all requests 503 — cache JWT validation, set tight authorizer timeout with fallback deny.
• Route misconfig: Blue/green route weight error — canary analysis on gateway metrics before full shift.
• Gateway god: 2000-line mapping templates — extract BFF, keep gateway thin.

Staff engineer insights

• Gateway is for cross-cutting edge policy — the moment you add business rules, you are building a BFF with extra steps.
• Amazon interview tip: distinguish API Gateway (edge routing) from service mesh (east-west) — candidates confuse them constantly.
• Cache JWT validation results at gateway — authorizer per request does not scale to Prime Day QPS.

Interview questions

Architecture review questions

• Are internal microservices unreachable from the public internet?
• Is authentication centralized at gateway with cached validation for hot paths?
• Are rate limits and WAF rules defined per route/client tier?
• Is business logic absent from gateway transforms (thin routing only)?
• Are OpenAPI specs the source of truth for request validation at edge?
• Are gateway metrics (latency, 4xx, 5xx, throttle) tied to client-facing SLOs?

Summary

API gateway at Amazon scale centralizes client-facing routing, Cognito auth, WAF protection, and VPC Link private integration so backend services evolve independently. Staff architects keep gateways policy-thin, cache auth decisions, and never embed domain logic at the edge.