Embeddings

Introduction

Embeddings are vectors (lists of numbers) that capture the meaning of text. They're produced by a smaller specialised model and used for semantic search, RAG, clustering and recommendation.

Beginner analogy: A meaning fingerprint — two sentences with the same idea get nearly identical fingerprints.

Understanding the topic

Core concepts to understand:

• Generated by embedding models (OpenAI text-embedding-3, Cohere, Voyage).
• Typical sizes: 384, 768, 1024, 1536, 3072 dimensions.
• Compared via cosine similarity.
• Stored in vector databases for fast search.

Syntax reference

Visual workflow / architecture:

"Open the door"   →  [0.12, -0.84, ..., 0.07]
"Unlock the entry"→  [0.13, -0.81, ..., 0.09]   ← very similar
"Pizza recipe"    →  [0.91,  0.10, ..., -0.42]  ← far away

cosine(a, b) close to 1 → same meaning

Real-world use

Powers Notion AI Q&A, Perplexity, Spotify song discovery, Pinterest visual search and every modern semantic-search feature.

Best practices

• Pick one embedding model and stick with it across your corpus.
• Normalise vectors (L2) for accurate cosine similarity.
• Re-embed if you upgrade models.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is an embedding?
• Q2. How do you compare two embeddings?
• Q3. What changes if you switch embedding models?