AI Optimization

Introduction

AI optimisation = making your features faster, cheaper, smarter without quality regression. The big wins come from caching, model routing, and prompt compression.

Beginner analogy: Like database query optimisation — small tweaks compound to massive savings.

Understanding the topic

Core concepts to understand:

• Prompt caching (Anthropic, OpenAI) — 50–90% off for repeated prefixes.
• Response cache for deterministic queries.
• Model routing — cheap model first, escalate when needed.
• Batching for embeddings and async work.
• Compression — strip whitespace, dedupe context.

Syntax reference

Visual workflow / architecture:

User query
   │
   ▼ deterministic? → response cache HIT → done
   │
   ▼ cheap model attempts answer
   │
   ▼ confidence high → done
   │
   ▼ escalate to premium model

Real-world use

Cursor, Perplexity, Notion AI all aggressively cache and route. Anthropic's prompt caching cuts costs 90% for long static contexts.

Best practices

• Always cache identical prompts.
• Use prompt caching for static system prompts.
• Route by complexity — most queries are easy.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is prompt caching?
• Q2. How does model routing save money?
• Q3. Compression techniques you'd use?