Semantic Search

Introduction

Semantic search finds results by meaning, not keywords. It's the front door of every RAG system and most modern search UIs.

Beginner analogy: Google understanding 'how do I send pictures from my phone' even if you typed 'iPhone MMS settings'.

Understanding the topic

Core concepts:

• Embed query, search vector DB, return top-k.
• Re-rank with cross-encoder for higher quality.
• Hybrid (BM25 + vector) catches both keywords and concepts.
• Tune k based on context budget.
• Filter by metadata (date, source, tenant).

Syntax reference

Visual workflow / architecture:

query ── embed ── ANN ── top-50
                              │
                              ▼
                         cross-encoder
                              │
                              ▼
                          top-5 → LLM

Real-world use

Notion search, Linear's natural-language filters, GitHub Copilot codebase chat.

Best practices

• Always re-rank for serious use cases.
• Show users which results were used.

Common mistakes

• No re-ranking → noisy results, frustrating users.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Semantic vs keyword search?
• Q2. What's a cross-encoder re-ranker?
• Q3. Scenario: vector search returns 50 results but only 5 are relevant. Fix?