Chain of Thought Prompting

Introduction

Chain-of-Thought (CoT) prompting asks the model to reason step-by-step before giving its answer. It dramatically improves performance on math, logic, multi-step reasoning and code-generation tasks.

Beginner analogy: Telling a student 'show your working' instead of just writing the final answer — they catch their own mistakes mid-way.

Understanding the topic

Core concepts to understand:

• Add 'Let's think step by step' or 'Reason carefully before answering'.
• Model verbalises intermediate steps → reaches correct answer more often.
• Works especially well on math, logic, multi-hop reasoning.
• Increases output tokens (cost) — but accuracy gains usually justify it.

Syntax reference

Visual workflow / architecture:

Without CoT:
  Q: A train leaves at 9am at 60 mph. Another at 10am at 80 mph.
     When do they meet?
  A: 1pm  ❌

With CoT:
  Q: ... Let's think step by step.
  A: At 10am the first train is 60 miles ahead.
     The second closes the gap at 20 mph.
     60 / 20 = 3 hours → they meet at 1pm. ✅

Real-world use

OpenAI's o1 and o3 models are essentially trained to do CoT internally. Chain-of-thought is the foundation of ReAct agents and Tree of Thoughts reasoning.

Best practices

• Use CoT for math, logic, planning and complex reasoning.
• For production, hide reasoning from end users (return only final answer).
• Combine with few-shot for best results.

Common mistakes

• CoT eats tokens — costly at scale.
• Model may produce wrong reasoning yet right answer — and vice versa.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is chain-of-thought prompting?
• Q2. Why does it improve accuracy on reasoning tasks?
• Q3. What's the cost trade-off?