Ambassador

Introduction

The ambassador pattern deploys a proxy container that simplifies outbound connections for the application — handling retries, circuit breaking, TLS, and protocol translation. Unlike the sidecar which handles both inbound and outbound, the ambassador focuses on egress simplification.

Uber uses ambassador-style proxies so application code makes simple localhost calls while the ambassador manages multi-datacenter routing, connection pooling, and mTLS to backend services.

Real production story

Uber's legacy Node.js services called downstream APIs directly with hardcoded endpoints and no retry logic. When the maps service migrated datacenters, 30 services needed code changes and redeploys. Introducing an ambassador proxy (localhost:9999) that handled service discovery, retries, and TLS termination let app code keep calling http://localhost:9999/maps — while the ambassador routed to the correct datacenter with circuit breaking. A misconfigured ambassador timeout (30s vs app's 5s expectation) caused thread pool exhaustion — teaching that ambassador and app timeout budgets must be aligned.

Business problem

Business pressure: Uber's microservices call dozens of internal APIs across datacenters. Direct coupling to endpoint URLs, TLS config, and retry logic in every service creates migration friction and inconsistent resilience behavior.

• Revenue at risk: Maps/routing API degradation cascades to trip matching — ambassador must circuit-break, not propagate timeouts.
• Engineering velocity: Datacenter migrations should not require N service redeploys — ambassador abstracts routing.
• Compliance / trust: mTLS to internal services must be consistent — ambassador centralizes cert management.

Architecture overview

An ambassador container acts as an outbound proxy in the pod. The application sends requests to the ambassador on localhost; the ambassador forwards to remote services with production-grade networking.

• Definition: Outbound proxy container simplifying external communication for the co-located app.
• When to adopt: Legacy services needing resilience and TLS without code changes; multi-datacenter routing.
• When to defer: Modern services with mature client libraries (gRPC with built-in LB) — ambassador adds latency.
• Operability: Ambassador error rate, upstream latency, circuit breaker state per destination.

Architecture motivation

Why architects care: The ambassador pattern decouples app code from network complexity. App calls localhost; ambassador handles discovery, TLS, retries, and load balancing. This is the egress half of the service mesh sidecar, usable standalone without full mesh.

• Force: Hundreds of services with heterogeneous languages calling shared internal APIs.
• Constraint: Cannot retrofit retry/TLS into every legacy service codebase quickly.
• Outcome: Standard ambassador image with service catalog integration and aligned timeout budgets.

Internal architecture

Uber ambassador egress topology — localhost proxy per pod:

• App sees single localhost endpoint — ambassador resolves real destination.
• Timeout budget: ambassador timeout < app timeout — app always gets response or fast fail.
• Circuit breaker on ambassador prevents retry storms to degraded upstreams.

Pod: trip-matching-service
├─ container: matching-app
│    outbound calls: http://127.0.0.1:9999/{service}/{path}
│    (never calls remote URLs directly)
└─ container: ambassador (Envoy/定制)
     listen: 127.0.0.1:9999
     routes:
       /maps/* → maps-service.uber.internal:443 (mTLS, retry 3x, timeout 2s)
       /pricing/* → pricing-service.uber.internal:443 (circuit breaker)
       /driver/* → driver-locator (datacenter-aware routing)
     service discovery: xDS from control plane
     connection pool: 100 per upstream

Data flow

Outbound: app HTTP/gRPC to localhost:9999 → ambassador resolves service → mTLS to upstream → response proxied back. Config update: control plane pushes route changes via xDS without app restart.

• Write path: App POST localhost:9999/maps/route → ambassador → maps-service with retry.
• Read path: Ambassador caches DNS/service catalog entries with TTL.
• Async path: xDS push updates routing during datacenter migration — zero app deploy.

// App code — simple localhost call (Node.js)
const MAPS_URL = "http://127.0.0.1:9999/maps";

async function getRoute(origin: LatLng, dest: LatLng) {
  const res = await fetch(`${MAPS_URL}/route`, {
    method: "POST",
    body: JSON.stringify({ origin, dest }),
    signal: AbortSignal.timeout(4000), // app timeout > ambassador timeout
  });
  return res.json();
}

// Ambassador Envoy config (simplified)
static_resources:
  listeners:
    - address: { socket_address: { address: 127.0.0.1, port_value: 9999 } }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                route_config:
                  virtual_hosts:
                    - name: maps
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/maps/" }
                          route:
                            cluster: maps-service
                            timeout: 2s
                            retry_policy: { num_retries: 3 }
  clusters:
    - name: maps-service
      type: STRICT_DNS
      circuit_breakers:
        thresholds: [{ max_connections: 100, max_retries: 10 }]
      transport_socket:
        name: envoy.transport_sockets.tls

System design diagram

Two diagrams show the Ambassador topology and the primary request/event path used in production at scale.

Production code example

Ambassador timeout budget enforcement — Uber platform config:

• Platform enforces timeout hierarchy at deploy time — not discovered during incident.
• Per-upstream circuit breaker thresholds derived from SLO error budget.
• xDS config changes audited with rollback to previous route version.

// Platform route template with enforced timeout hierarchy
const routeDefaults = {
  maps: { ambassadorTimeoutMs: 2000, appTimeoutMs: 4000, retries: 3 },
  pricing: { ambassadorTimeoutMs: 1500, appTimeoutMs: 3000, retries: 2 },
  driver: { ambassadorTimeoutMs: 1000, appTimeoutMs: 2500, retries: 2 },
};

// Validation: reject deployment if app timeout < ambassador timeout
function validateTimeoutBudget(appConfig: AppConfig): void {
  for (const [service, budget] of Object.entries(routeDefaults)) {
    const appTimeout = appConfig.timeouts[service];
    if (appTimeout <= budget.ambassadorTimeoutMs) {
      throw new Error(
        `App timeout (${appTimeout}ms) must exceed ambassador timeout (${budget.ambassadorTimeoutMs}ms) for ${service}`
      );
    }
  }
}

Enterprise case study

Uber — ambassador proxy for legacy service egress: Platform injected ambassador container into pods of services not yet on full service mesh. Datacenter migration of maps service required zero app redeploys — xDS route update only.

• Before: 30 services hardcoded maps endpoint; datacenter move = 30 redeploys.
• Decision: Mandatory ambassador for outbound calls in legacy services; timeout budget documented.
• After: Datacenter migrations via xDS push; circuit breaker prevented maps outage from cascading to matching.

Trade-offs

• Simplicity vs latency: Extra hop through ambassador adds ~1ms — acceptable for most; not for ultra-low-latency paths.
• Ambassador vs client library: Ambassador works across languages; gRPC client LB is tighter for gRPC-only services.
• Standalone vs service mesh: Ambassador is egress-only sidecar; full mesh adds inbound policy too.
• Timeout alignment: Misaligned timeouts cause thread exhaustion — document timeout budget chain.

Security considerations

Ambassador holds mTLS identity: Compromised ambassador can reach all configured upstreams — restrict route table and rotate certs via SDS.

• Identity: Ambassador presents service identity cert to upstreams — app never holds TLS keys.
• Data: Ambassador can inspect plaintext on localhost — acceptable within pod; mTLS on egress.
• Supply chain: Pin ambassador image; route table changes audited in control plane.

Scalability analysis

Scale dimensions: Uber matching service makes 50+ outbound calls per trip request. Ambassador connection pool sizing and circuit breaker thresholds dominate.

• Horizontal scale: Ambassador scales 1:1 with pods; tune connection pool per upstream.
• Hot spots: Maps service degradation — ambassador circuit breaker protects matching pods from retry storms.
• Cost: Ambassador CPU scales with outbound QPS — profile before mandating on batch workloads.

Failure scenarios

What breaks: Ambassador timeout shorter than app causes premature errors; circuit breaker open blocks all calls including healthy retries; stale xDS routes to decommissioned datacenter.

• Timeout mismatch: Ambassador 2s timeout, app waits 10s — app threads blocked on dead connections — align budgets.
• Circuit breaker flapping: Threshold too aggressive — tune per-upstream based on historical error rate.
• Stale routes: xDS disconnect — ambassador serves last-known-good; alert on config age > 60s.

Staff engineer insights

• Ambassador is the egress sidecar — use when app code cannot own retry/TLS logic.
• Timeout budget chain: upstream SLA < ambassador timeout < app timeout — always.
• Circuit breaker thresholds per upstream based on error budget, not defaults.
• Datacenter migration is the killer use case — xDS route update beats N redeploys.

Interview questions

Architecture review questions

• App calls localhost ambassador, not remote URLs directly?
• Timeout budget: ambassador < app timeout documented and enforced?
• Circuit breaker per upstream with tuned thresholds?
• mTLS configured on ambassador egress?
• xDS config age monitored with alert?
• Route changes tested in shadow before production push?

Summary

The ambassador pattern at Uber scale means localhost egress proxies with aligned timeout budgets, per-upstream circuit breakers, and xDS-driven routing for zero-downtime datacenter migrations. Use when app code should not own network resilience — but operate the ambassador with the same rigor as any critical proxy.