What is RAG?

Introduction

Retrieval-Augmented Generation (RAG) is the most important pattern in production AI. Instead of asking an LLM to answer from memory (which causes hallucinations), you retrieve relevant documents first and pass them as context.

Beginner analogy: Closed-book exam = naked LLM. Open-book exam = RAG. Open-book always wins.

Understanding the topic

Core concepts to understand:

• Index documents → embeddings → vector DB.
• At query time: embed the question → search vectors → grab top-k docs.
• Pass docs + question to LLM → grounded answer.
• Hallucinations drop, factual accuracy soars.
• Knowledge stays fresh — just re-index, no re-training.

Syntax reference

Visual workflow / architecture:

Indexing (offline):
   docs → chunks → embeddings → Vector DB

Query time:
   question → embed → search → top-k chunks
              │                       │
              └────────► LLM ◄────────┘
                          │
                          ▼
                    Grounded answer (with citations)

Real-world use

Perplexity, Notion AI Q&A, ChatGPT 'browse the web', Glean, Vercel docs Q&A — all are RAG.

Best practices

• Always cite sources back to the user.
• Chunk smartly — paragraphs, not arbitrary character splits.
• Re-rank top-k results before sending to LLM.
• Evaluate retrieval and generation separately.

Common mistakes

• Bad chunking = bad retrieval = bad answers.
• Returning too many docs → context overflow + cost.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is RAG and why does it help?
• Q2. Walk through indexing and query time.
• Q3. Why include citations?