Choosing the Right Model

Introduction

Model selection is the most under-discussed prompt-engineering decision. The right model can halve cost, improve accuracy, or unlock a feature.

Beginner analogy: Like choosing the right tool — screwdriver vs drill vs lathe.

Understanding the topic

Core concepts to understand:

• Decision axes: accuracy, cost, latency, context length, multimodal, privacy.
• Frontier (GPT-4o, Claude 3.5, Gemini 1.5 Pro): hardest tasks.
• Mid-tier (Haiku, Mini, Flash): routing & bulk.
• Open-source (Llama, Mistral): self-host & cost.
• Re-evaluate quarterly — pricing & capability shift fast.

Syntax reference

Visual workflow / architecture:

Task type           Default model
─────────────────────────────────────
Bulk classify       Haiku / Mini
Code refactor       Claude 3.5 Sonnet
Research summary    Gemini 1.5 Pro
Realtime chat       GPT-4o Mini stream
Sensitive data      Self-hosted Llama

Real-world use

Klarna, Shopify, Stripe — all run multi-model stacks with quarterly model reviews.

Best practices

• Run your eval set against multiple models before committing.
• Track model performance per feature.
• Build for swap-ability (LiteLLM, OpenRouter).

Hands-on exercise

Interview preparation — practice these questions:

• Q1. How choose a model?
• Q2. Why swap-ability matters?
• Q3. Re-evaluation cadence?
• Q4. Cost vs quality tradeoff?