Document Chunking

Introduction

Chunking splits documents into smaller pieces before embedding. Bad chunking is the #1 cause of bad RAG. Get this right.

Beginner analogy: Like cutting a pizza — too big and people can't eat a slice; too small and the toppings fall off.

Understanding the topic

Core concepts to understand:

• Aim for 200–800 tokens per chunk.
• Preserve semantic boundaries (paragraphs, headings).
• Overlap chunks slightly (10–20%) to avoid losing context at edges.
• Add metadata: source URL, heading hierarchy, page #.
• Different content needs different strategies (code, tables, prose).

Syntax reference

Visual workflow / architecture:

Document (10,000 tokens)
       │
       ▼  split by headings + 500-token max
   Chunk 1 (500 tok) [+ metadata]
   Chunk 2 (500 tok) [+ metadata]
   ...
       │
       ▼  embed each
   Vector DB

Real-world use

Frameworks like LangChain and LlamaIndex ship dozens of chunkers. Unstructured.io handles complex PDFs/HTML.

Best practices

• Chunk by heading + sentence — never by raw character count.
• Add overlap to avoid losing context at edges.
• Store rich metadata for filtering and citation.

Common mistakes

• 500-character splits cut sentences in half.
• Forgetting to handle tables, lists, code differently.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Why does chunking matter so much?
• Q2. What's a good chunk size?
• Q3. Why use overlap?