RAG Optimization

Introduction

Production RAG optimization turns a mediocre demo into a stellar product. The big levers: chunking strategy, hybrid search, re-ranking, query rewriting, multi-vector, and evaluation.

Beginner analogy: a librarian who knows your habits and asks clarifying questions — much better than one who throws random books at you.

Understanding the topic

Core concepts:

• Chunking: structure-aware (sections, code blocks).
• Hybrid: BM25 + vector union.
• Re-ranking: cross-encoder on top-50 → top-5.
• Query rewriting: expand ambiguous queries.
• Multi-vector: HyDE, ColBERT.
• Always measure with a golden eval set.

Syntax reference

Visual workflow / architecture:

raw → rewrite → embed
                 │
                 ▼
            hybrid search
                 │
                 ▼
            re-rank → top-5
                 │
                 ▼
              LLM (cite)

Real-world use

Perplexity, Glean and Copilot codebase chat all stack these optimisations.

Best practices

• Build a golden eval set first.
• Iterate one optimisation at a time.

Common mistakes

• Stacking 5 'best practices' at once — can't tell what helped.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Five RAG optimisations.
• Q2. What does HyDE solve?
• Q3. Scenario: golden eval drops after a chunking change. What now?