AI Workflow Optimization

Introduction

Workflow optimization = remove steps, parallelise others, cache repeats, swap models, batch where possible. Apply continuously based on traces.

Beginner analogy: Toyota Production System — measure, eliminate waste, repeat.

Understanding the topic

Core concepts:

• Identify hottest, slowest, most expensive nodes from traces.
• Cut steps that don't move quality.
• Parallelise independent steps.
• Cache deterministic node outputs.
• Swap models per node based on eval.

Syntax reference

Visual workflow / architecture:

trace ─► find slow/expensive node
        │
        ▼
   try: cache · cheaper model · skip · parallel
        │
        ▼
   re-eval ──► keep winner

Real-world use

Cursor optimises autocompletion latency monthly; Devin tunes step counts per task class.

Best practices

• Optimise based on traces, never guesses.
• One change at a time; eval each.

Common mistakes

• Bulk 'optimisations' that introduce regressions you can't isolate.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Three optimisation levers.
• Q2. Why isolate changes?
• Q3. Scenario: latency p95 doubled overnight. What artefact do you open first?