How LLMs Work

Introduction

Large Language Models (LLMs) like GPT-4, Claude, Gemini and Llama predict the next token (a word piece) given the previous tokens. By doing this on trillions of tokens, they learn grammar, facts, reasoning patterns and code. That single skill — next-token prediction — is enough to power every agent you'll ever build.

Beginner analogy: autocomplete on your phone, but trained on the entire internet and books. The agent on top tells it what to autocomplete.

Understanding the topic

Core concepts:

• Input is tokenised into ~3-4 characters per token.
• The model outputs logits — probability over every token in its vocabulary.
• Sampling (temperature, top-p) picks the next token.
• The model is stateless — context = the whole prompt every call.
• Context window limits how much fits (8k → 1M+ tokens depending on model).

Syntax reference

Visual workflow / architecture:

prompt: "The cat sat on the"
   │ tokenize
   ▼
[The][ cat][ sat][ on][ the]
   │
   ▼ transformer attention
   │
logits → P(mat)=0.7, P(rug)=0.2, P(table)=0.05 …
   │ sample
   ▼
next token: " mat"
   │ append
   ▼
"The cat sat on the mat" → repeat

Real-world use

GPT-4 powers ChatGPT and Copilot; Claude powers Cursor; Gemini powers Google Workspace; Llama powers Meta AI and most open-source agents.

Best practices

• Learn temperature: 0 = deterministic, 1+ = creative.
• Mind the context window — pay for every token in and out.
• Cache prompts when possible (Anthropic & OpenAI both support it).

Common mistakes

• Treating the LLM as omniscient — it's a next-token predictor, not a database.
• Forgetting the context limit until production errors.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What does an LLM actually predict?
• Q2. What is temperature?
• Q3. Why are LLMs stateless?
• Q4. Difference between context window and training data?
• Q5. Scenario: your agent keeps cutting off mid-answer. What's wrong?