Context Management

Introduction

The context window is the limited workspace an LLM sees per call. In long agent runs, context can balloon to thousands of messages. Managing it is the difference between an agent that works for 5 turns and one that runs all day.

Beginner analogy: a chef's prep table — you can only fit so much; smart chefs prep in batches and clear what they don't need.

Understanding the topic

Core concepts:

• Sliding window: keep last N turns.
• Summarisation: collapse old turns into a summary.
• Retrieval: pull only relevant facts from a vector store.
• Tool call truncation: trim huge tool outputs.
• Cache stable prefix (system + tools) to make recall free.

Syntax reference

Visual workflow / architecture:

[ System prompt ][ Tool docs ][ Summary of turns 1-20 ][ Last 5 turns ][ New user msg ]
        ▲ cached ▲       ▲ cached ▲     ▲ rolling ▲              ▲ live ▲

Real-world use

Cursor, Devin and Operator all use aggressive context management — summarising past steps, indexing files, caching system prompts.

Best practices

• Measure tokens per turn — graph it.
• Summarise once, store the summary; don't re-summarise every turn.

Common mistakes

• Letting context grow unbounded — eventually hits the limit and breaks.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Four techniques to manage context?
• Q2. Why does context bloat hurt cost AND quality?
• Q3. Scenario: your agent forgets a key user fact halfway through. How do you fix it?