AI Engineering Scenarios

Introduction

Scenario-based questions test your judgment under realistic constraints. Practise reasoning out loud through these.

Understanding the topic

Core concepts to understand:

• AI feature launches viral. Cost spikes 50×. What do you do?
• Customer reports the AI leaked another customer's data. What now?
• New model from OpenAI is better but 2× slower. Roll out?
• Hallucination caused a real-world incident. Postmortem plan?
• Provider goes down for 6 hours. How do you keep the product working?
• Your eval set is 'too easy' and prod accuracy is dropping. Plan?

Syntax reference

Visual workflow / architecture:

Scenario
  │
  ▼
Clarify (scope, scale, severity)
  │
  ▼
Triage (immediate mitigations)
  │
  ▼
Root cause + permanent fix
  │
  ▼
Post-mortem + eval to prevent recurrence

Real-world use

These are real on-call incidents at every AI company.

Best practices

• Always triage before fixing.
• Communicate with users transparently.
• Add an eval / guardrail to prevent recurrence.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Walk through the cost-spike scenario.
• Q2. How do you handle the data-leak scenario?
• Q3. How would you migrate to a slower-but-better model?