Video Streaming Platform

Introduction

Design a video streaming platform like YouTube or Netflix — upload, transcode, store, and deliver video globally with adaptive bitrate playback. HLD emphasizes blob storage, CDN egress, encoding pipeline, and metadata catalog separate from bytes.

Upload bandwidth, transcoding CPU farm, and CDN cost dominate. Assume millions of videos, popular 1% driving 80% views, multiple resolutions (360p–4K).

Understanding the topic

Key concepts

• Upload: multipart to S3 presigned URL; metadata row CREATED.
• Transcode: worker pulls raw, outputs HLS/DASH segments multiple bitrates.
• Storage: S3 for objects; CDN caches segments at edge.
• Playback: client requests manifest (.m3u8) then adaptive segment fetches.
• Catalog: PostgreSQL/Elasticsearch title search separate from video blobs.
• DRM optional Widevine for premium content.

flowchart TB
  Upload --> Transcode
  Transcode --> S3
  CDN --> Player
  Player --> Auth

Internal architecture

Architecture overview

flowchart TB
  Upload --> Transcode
  Transcode --> S3
  CDN --> Player
  Player --> Auth

Step-by-step explanation

1. POST /videos/init → presigned S3 URL + videoId.
2. S3 event → Kafka → Transcode workers (FFmpeg) → output bucket /videoId/{bitrate}/seg.ts.
3. Catalog service marks READY; invalidate CDN manifest cache.
4. GET /watch/{id} → auth check → return manifest URL (CDN signed).
5. Player ABR switches bitrate by bandwidth estimate.
6. Popular videos pre-warm CDN; cold content origin fetch on first view.

Informative example

Initiate upload and transcode job enqueue on S3 complete event:

@RestController
@RequestMapping("/api/v1/videos")
public class VideoController {
    private final VideoService videos;

    public VideoController(VideoService videos) { this.videos = videos; }

    @PostMapping("/init")
    public UploadInitResponse init(@RequestBody UploadInitRequest req, @AuthenticationPrincipal Jwt jwt) {
        return videos.initUpload(jwt.getSubject(), req.title(), req.contentType());
    }
}

@Service
public class VideoService {
    public UploadInitResponse initUpload(String userId, String title, String contentType) {
        String videoId = UUID.randomUUID().toString();
        db.insertPending(videoId, userId, title);
        URL presigned = s3.presignPut("raw/" + videoId, contentType, Duration.ofHours(1));
        return new UploadInitResponse(videoId, presigned.toString());
    }

    @KafkaListener(topics = "s3.upload.completed")
    public void onUploadComplete(UploadEvent event) {
        kafka.send("transcode.jobs", event.videoId(), new TranscodeJob(event.videoId(), List.of("360p","720p","1080p")));
    }
}

Transcode is async CPU farm — auto-scale on Kafka lag. CDN serves 95%+ bytes.

Real-world use

Real-world use cases

• OTT subscription Netflix model.
• User-generated YouTube ad-supported model.
• Enterprise training video LMS.
• Live sports low-latency HLS variant.

Best practices

• Never serve video through app servers — S3 + CDN only.
• Chunk upload resume for mobile flaky networks.
• Store multiple bitrates for ABR.
• Signed URLs/time-limited cookies for premium.
• Monitor CDN hit ratio and origin egress cost.
• Content moderation ML pipeline async on upload complete.

Common mistakes

• Single bitrate — poor mobile experience.
• Transcode synchronous in upload request — timeout.
• No CDN — origin bandwidth bankruptcy.
• Metadata and blob tight coupling — search scaling blocked.
• Ignoring copyright ID scan before publish.

Advanced interview questions

Summary

Video HLD separates upload/transcode pipeline from CDN delivery. S3 object storage + CDN edge serves adaptive HLS/DASH. Async transcode workers scale on queue depth. Catalog metadata in SQL/ES independent of blobs. Signed URLs protect premium streams. Ride booking adds real-time matching similar to food dispatch.