Introduction to RAG

Introduction

Retrieval-Augmented Generation (RAG) grounds an LLM in your private data by retrieving relevant chunks at query time and stuffing them into the prompt. RAG is how agents 'know' facts beyond training data.

Beginner analogy: open-book exam — the student (LLM) doesn't memorise; they look up the right page (chunk) and answer.

Understanding the topic

Core concepts:

• Index documents → embed → store in vector DB.
• At query time: embed query → top-k search → inject into prompt.
• Reduces hallucinations and keeps knowledge fresh.
• Cheaper than fine-tuning for most use cases.
• Foundation of nearly every enterprise agent.

Syntax reference

Visual workflow / architecture:

 Question
    │
    ▼
┌──────────────┐
│  Embed Query │
└──────┬───────┘
       ▼
┌──────────────┐      ┌──────────────┐
│ Vector Store │◄────►│  Documents   │
└──────┬───────┘      └──────────────┘
       │ top-k chunks
       ▼
┌──────────────┐
│   LLM + ctx  │
└──────┬───────┘
       ▼
  Grounded Answer

Real-world use

ChatGPT 'browse with Bing', Cursor codebase chat, Perplexity, Notion AI Q&A, Glean — all RAG systems.

Best practices

• Chunk smart — paragraphs, not arbitrary windows.
• Re-rank top-50 with a cross-encoder for quality.
• Always show citations.

Common mistakes

• Stuffing too many chunks — context bloat hurts quality.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Why RAG over fine-tuning?
• Q2. Three steps of the RAG flow?
• Q3. Scenario: RAG returns irrelevant chunks. Three fixes?