Chain of Thought Prompting

Introduction

Chain of Thought (CoT) asks the model to think step by step before answering. It dramatically improves accuracy on math, logic, and multi-step reasoning — sometimes by 30+ points.

Beginner analogy: Like showing work in math class. Forcing intermediate steps catches mistakes that a one-shot answer hides.

Understanding the topic

Core concepts to understand:

• Add: 'think step by step before answering'.
• Best for math, logic puzzles, multi-step reasoning.
• Improves accuracy at the cost of more tokens & latency.
• Pairs perfectly with few-shot reasoning examples.
• Modern 'reasoning models' (o1, o3, Claude 3.7) do CoT internally.

Syntax reference

Visual workflow / architecture:

User: "If Anna has 3 apples and Bob 5, and they share..."

Without CoT → "4 each" (wrong)
With CoT → "Step 1: total = 8.
              Step 2: share = 8/2 = 4 each.
              Answer: 4 each." (right)

Real-world use

GPT-3.5 jumped from 17% to 78% on grade-school math when CoT was used — single largest accuracy win in the original CoT paper.

Best practices

• Trigger phrases: 'Let's think step by step.', 'First, ... Then, ...'.
• Place CoT instruction BEFORE the question.
• Use silent CoT in production: hide reasoning, return only the final answer.

Common mistakes

• CoT on simple tasks wastes tokens & can introduce errors.
• Exposing raw reasoning to end users (verbose & confusing).

Hands-on exercise

Interview preparation — practice these questions:

• Q1. What is CoT and when does it help?
• Q2. Why does CoT improve accuracy?
• Q3. Difference between explicit and implicit (reasoning model) CoT?
• Q4. How do you hide CoT from end users?