Orchestration Patterns

Single agent by default; add an orchestrator, a handoff, or parallelism only when a specific constraint forces it.

Agentic Engineering · Advanced · ~6 min

In plain English

Choosing the shape of the work: a straight pipeline, a branch, a loop, or several specialists. Pick the simplest one that can express the task.

Why it's worth your time

Most agent complexity is self-inflicted — a fixed workflow would have done the job, faster and testably.

If you remember three things

Chain for known steps, router for known branches, loop only for genuinely open paths
Parallelize independent work; barriers cost wall-clock
Simplest shape that works, always

Overview

The default architecture is one agent with one context, and it wins more often than the diagrams suggest: nothing is lost in a handoff, no worker duplicates work, and one trace explains the whole run. Orchestration is what you reach for when a specific constraint appears. Independent, read-only subtasks that dominate wall-clock justify an orchestrator fanning out to workers — at the cost of roughly N× the tokens and a merge step. Distinct specialisations or permission boundaries justify routing, where control transfers with the state rather than splitting. And dependencies decide sequencing: if step N needs step N−1's output, no amount of concurrency makes it correct. The engineering judgment is knowing which constraint you actually have.

How it works

One agent, one thread The baseline: one context, no handoff loss, one trace. Often the right answer.
Orchestrator and workers Fan out independent read-only work. Wall-clock drops; tokens rise; a merge step appears.
Handoffs and routing Control transfers to a specialist. The handoff must carry goal, attempts, evidence and budget.
Parallel vs sequential Dependencies decide. If step N needs N−1's output, concurrency just produces confident wrong results.
When simple wins The mesh loses on cost, moving parts, and traces-per-incident. Need a reason before you add an agent.

In an interview

I start with one agent and one context, because nothing is lost in a handoff and one trace explains the run. I add orchestration only for a named constraint. Independent, read-only subtasks that dominate wall-clock justify an orchestrator fanning out to workers — but that costs roughly N× the tokens plus a merge step, so it's money for latency. Specialisation or a permission boundary justifies routing, where control transfers and the handoff must carry the goal, what's been tried, the evidence and the remaining budget. Dependencies decide sequencing: if step N needs N−1's output it stays a chain. Otherwise, one well-built agent with sharp tools and real evals beats the mesh.

Production defaults

Default: a chain. One LLM call per step, deterministic control flow, fully testable
Router: when there are a few known paths and a cheap classifier can pick
Open loop: only when the path genuinely varies per request
Parallel: fan out independent sub-tasks; only barrier when a step truly needs all prior results

What breaks

Non-deterministic behaviour on the same input — An open loop where a chain would do. Fix the shape before tuning the prompt.
Slow despite parallel calls — Barriers between stages. Pipeline items independently instead of synchronizing every stage.

Watch it explained

Conceptual Guide: Multi Agent Architectures — LangChain, 8:58