Embeddings

Introduction

Embeddings are dense vectors that capture semantic meaning. Similar meaning → similar vectors. They power RAG, semantic search, recommendations and clustering.

Beginner analogy: a map of meaning — 'dog' and 'puppy' live next door; 'dog' and 'banana' are on opposite continents.

Understanding the topic

Core concepts:

• Models: text-embedding-3-small / large, Voyage, Cohere, Nomic.
• Typical dimensions: 384 → 3072.
• Compare with cosine similarity.
• Re-embed when you change models.
• Domain-specific embeddings (code, legal) can beat general models.

Syntax reference

Visual workflow / architecture:

"refund my order"  ──► [0.12, …, -0.03]
"return my purchase" ──► [0.10, …, -0.05]
        cosine ≈ 0.94 (very similar)

Real-world use

Every modern search box (Notion, Linear, Slack) uses embeddings under the hood.

Best practices

• Benchmark embedding models on your data (MTEB-style).
• Store vectors in dedicated DBs (pgvector, Pinecone, Qdrant).

Common mistakes

• Mixing embeddings from different models in one index.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is an embedding, plain English?
• Q2. Why cosine similarity?
• Q3. Scenario: domain accuracy is poor. Two options?