Context Windows

Introduction

The context window is the maximum number of tokens an LLM can read in a single call (prompt + response combined). Bigger context = more knowledge per call, but more cost and slower latency.

Beginner analogy: Like RAM for the model — it can only think about what fits in memory at once.

Understanding the topic

Core concepts to understand:

• GPT-4o: 128k tokens. Claude 3.5 Sonnet: 200k. Gemini 1.5 Pro: 1M+.
• 1k tokens ≈ 750 English words.
• Cost & latency scale with input size.
• Lost in the middle — models often miss content buried mid-context.

Syntax reference

Visual workflow / architecture:

[ system | retrieved context | history | user msg | response ]
 ←──────── context window (e.g. 128k tokens) ────────→

Cost = (input_tokens × in_price) + (output_tokens × out_price)

Real-world use

Long-context models enable whole-codebase Q&A (Cursor, Sourcegraph), long-document summarisation, and multi-document RAG.

Best practices

• Don't fill the window 'just because' — it slows everything down.
• Place critical info at the start or end (avoid the middle).
• Cache static parts of the prompt (Anthropic + OpenAI both support this).

Common mistakes

• Assuming 1M context means perfect recall — accuracy drops with length.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is a context window?
• Q2. What is 'lost in the middle'?
• Q3. How do you keep cost under control with long contexts?