AI Cost Optimization

Introduction

Cost optimization = matching model + prompt + caching strategy to each step. Done well, you halve bills without quality loss.

Beginner analogy: hiring — don't put a senior partner on every task; route simple jobs to interns.

Understanding the topic

Core concepts:

• Pick the cheapest model that passes the step's eval.
• Prompt caching for stable prefixes.
• Truncate / summarise long context.
• Batch eligible calls.
• Use cheaper embeddings where quality allows.

Syntax reference

Visual workflow / architecture:

┌──────────┐    ┌──────────┐    ┌──────────┐
│ Step A   │──► │ Step B   │──► │ Step C   │
│ Haiku    │    │ Sonnet   │    │ Opus     │
│ $0.001   │    │ $0.02    │    │ $0.20    │
└──────────┘    └──────────┘    └──────────┘

Real-world use

Cursor uses Haiku for autocompletion, Sonnet for chat, Opus for hard tasks — the same pattern you should copy.

Best practices

• Measure cost per step, not per run.
• Promote cached prompts aggressively.

Common mistakes

• Using the smartest model for every step — 5-10x overspend.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Three cost levers.
• Q2. How does prompt caching work?
• Q3. Scenario: bill is 3x budget. Which step do you check first?