The Coordination Tax: When Multi-Agent Systems Are Worth It

I was charged twice for order 4187. One package is still missing, and I need the duplicate charge reversed.

A capable support agent could investigate this ticket, check policy, propose a resolution, carry out an approved action, and draft a response. Using five agents adds five places to lose context, repeat errors, or issue the same refund twice.

Multiple agents can add expertise, parallelism, and separation of authority. They also add handoffs, model calls, shared-state problems, latency, cost, and failure paths.

A multi-agent system is justified only when measurable gains from specialization, parallelism, or trust isolation exceed its coordination tax.

The coordination tax

Collaboration means several agents contribute to one outcome. Coordination allocates work, sequences dependencies, shares state, resolves conflicts, and controls side effects. Its tax includes extra latency and cost, handoffs, synchronization, duplicate work, verification, recovery, and error propagation.

Here, \(G\) means measurable gain and \(C\) means coordination cost.

This is a decision rule, not a literal equation. For each workload, compare expected gains in quality, speed, or risk reduction with the added coordination cost. An extra model call may be worthwhile if parallel work reduces latency. Separating payment recommendation from execution may be worthwhile even when it adds latency, because it reduces risk.

The rule applies only when another agent is actually needed. A workflow does not become multi-agent simply because it has several steps or model calls. Add another agent only when the role needs distinct context, tools, evaluation criteria, authority, or lifecycle.

Ask: Which measured bottleneck requires another agent?

What the evidence says

Research supports this decision rule with five practical findings:

1. Task structure dominates agent count. Across 260 configurations, the best multi-agent results ranged from an 80.8% gain on decomposable financial reasoning to a 70% loss on sequential planning. Tool-heavy tasks added overhead, and systems without centralized verification propagated more errors. Architecture must match task dependencies (Kim et al.).
2. Useful diversity beats replicas. Two diverse agents could match or exceed 16 homogeneous agents. Diversity must add independent evidence, tools, or methods—not cosmetic personas (Yang et al.).
3. Communication is not reasoning. In 1,620 SILO-BENCH experiments, agents exchanged messages but often failed to combine distributed information. On the hardest tasks, success fell to zero beyond 50 agents. Conversation cannot replace a data model and task graph (Zhang et al.).
4. Structured multi-agent reasoning can outperform repeated sampling. At equal compute, debate and mixture-of-agents scored 1.3 and 2.7 percentage points higher than self-consistency—a method that generates several independent solutions with one model and selects the most common answer. This result applies to repeated model calls on a single reasoning problem. It does not show that production workflows with tools, shared state, and real-world actions are more reliable (Wunderlich et al.).
5. Coordination needs explicit limits and evaluation. Analysis of more than 1,600 traces found 14 failure modes across system design, agent alignment, and verification (Cemri et al.). In math problem solving, gains diminished beyond roughly ten agents and noisy inputs still caused a robustness gap (Alavi et al.). In open-ended work, dense communication caused premature convergence (Chen et al.). For production systems, start with a small team, limit message fan-out, test noisy inputs, and evaluate the full execution trace—not only the final answer.

Use multiple agents when they can work independently in parallel or bring different information. Validate each result before another agent uses it. Prefer one agent when work must happen in order, requires many tool calls, updates shared data, or depends on long conversations between agents.

Three different systems

The label “multi-agent” covers three different designs. Their evidence and risks are not interchangeable:

• Model ensembles generate several answers to one problem, then debate, critique, vote, or combine them. They use extra model calls to improve one answer; they do not test production workflow coordination.
• Orchestrated workflows give agents different context, tools, or permissions. A task plan and one shared record control the work. This is the main production pattern discussed here.
• Distributed ecosystems connect systems operated by different teams or companies. MCP connects models to tools and data; A2A lets agent systems communicate. Neither protocol tells you how to organize the work or whether another system can be trusted.

A result from one design does not automatically apply to another.

Start with one agent

Start with the simplest design: one agent. For the support ticket, give it read-only data access, policy search, a structured case record, a controlled action interface, and human approval for risky actions. Measure whether it resolves cases correctly, follows policy, avoids duplicate or unauthorized actions, and meets latency, cost, and review-time targets.

Add an agent for only three reasons:

• Specialization: Another role needs different context, tools, or evaluation criteria. A reviewer helps only when it uses different evidence or a different validation method. Two agents can agree on the same mistake.
• Parallelism: Two tasks do not depend on each other, and running them together reduces total time. Conversation that adds no speed is overhead.
• Trust isolation: Different permissions reduce risk. An agent may propose a refund, but a controlled gateway validates and executes it within limits. This separation can justify more latency.

Do not split work to mirror an organization chart or create intuitive personas. If one agent meets the targets, stop.

Evolve the design one step at a time

Do not design the final multi-agent system upfront. Start with one agent, measure its failures, and add only the component that addresses the next measured problem.

Step 1: One agent

One support agent reads the case, checks policy, proposes a resolution, and uses a controlled action tool. Human approval remains required for risky actions.

flowchart LR
    accTitle: Step 1 single-agent support workflow
    accDescr: One support agent handles case facts, policy, and resolution. It uses a controlled action tool, with human approval for risky actions, before sending the customer response.

    ticket["Customer ticket"] --> support["Single support agent<br/>Facts · policy · resolution"]
    support --> action["Controlled action tool<br/>Human approval for risky actions"]
    action --> response["Customer response"]

Benchmark this design. If it meets the quality, speed, cost, and safety targets, stop.

Step 2: Split read-only work

Suppose the policy library and transaction history make the agent less accurate, and sequential lookups cause most of the delay. First try parallel tool calls inside the same workflow. If separate contexts still improve quality, add two read-only agents: one for case facts and one for policy. Run them in parallel and keep the original support agent for the final resolution.

flowchart TD
    accTitle: Step 2 parallel read-only specialists
    accDescr: A workflow controller sends the ticket to a facts agent and a policy agent in parallel. Their results go to the original support agent, which creates the resolution and uses the existing controlled action tool.

    ticket["Customer ticket"] --> controller["Workflow controller"]
    controller --> facts["Facts agent<br/>Read-only transaction tools"]
    controller --> policy["Policy agent<br/>Read-only policy sources"]
    facts --> support["Original support agent<br/>Creates resolution"]
    policy --> support
    support --> action["Controlled action tool"]
    action --> response["Customer response"]

Benchmark again. Keep the split only if it improves quality or latency enough to cover the extra calls and handoffs.

Step 3: Separate decisions from execution

Suppose the remaining risk is that the same agent can recommend and execute a refund. Remove its execution permission. It now submits a structured proposal to a gateway that enforces limits, executes the action once, or sends the case to a human.

flowchart TD
    accTitle: Step 3 isolated action execution
    accDescr: Read-only facts and policy agents send evidence to a shared case record. The original support agent proposes a resolution without execution permission. A gateway executes allowed actions once or sends risky cases to a human. The response waits for the confirmed result.

    ticket["Customer ticket"] --> controller["Workflow controller<br/>Case ID · task dependencies · deadlines"]
    controller --> facts["Facts agent<br/>Read-only"]
    controller --> policy["Policy agent<br/>Read-only"]
    facts --> record["Versioned case record<br/>Evidence · sources · state version"]
    policy --> record
    record --> resolution["Original support agent<br/>Proposes resolution · cannot execute"]
    resolution --> gateway["Action gateway<br/>Request format · limits · execute once · audit"]
    gateway -->|Allowed| execute["Execute action"]
    gateway -->|Risky or unclear| review["Human review"]
    execute --> response["Response after confirmed result"]
    review --> response

Each step must solve an observed problem. Re-test after every change and stop as soon as the system meets its targets.

Use agents for judgment. Use code for scheduling, input validation, permissions, safe retries, and actions. Keep these boundaries:

• Workflow controller: Creates the case ID, task dependencies, deadlines, retry budget, and idempotency key, which prevents the same action from running twice.
• Versioned case record: Stores the current state, evidence, and sources. A chat transcript is not system state.
• Facts and policy agents: Return structured findings, source links, and the state version they read.
• Resolution agent: Joins facts and policy into a proposal but cannot issue a refund.
• Action gateway: Checks the request, enforces limits, executes it at most once, and records executed, denied, or pending.
• Human review: Handles ambiguity, conflicting evidence, and high-risk actions.

Write the customer response only after recording the final action result.

Coordination failures happen at boundaries

This design adds useful controls, but every handoff creates another place to fail.

An investigator may mistake a payment authorization for a settled charge. The resolution agent may then recommend a refund. If the refund succeeds but the action call times out, a retry may issue a second refund. The failure is not one bad answer. It is a chain of incomplete data, stale state, unsafe retries, and excessive authority.

Use code—not another language model—to enforce one-time execution, limits, and access rules. Use a reviewer only when it has different evidence, a different method, or separate authority. Send unresolved cases to a human.

Prove that the team earns its complexity

These controls reduce risk, but extra agents still have to prove their value. Test three questions:

1. Does another agent improve the result? Measure quality, speed, or risk reduction. “More perspectives” is not enough.
2. Can you control coordination? Every task needs an owner, structured output, dependencies, and one authoritative record. Every action needs limited permissions, safe retries, escalation, and a trace.
3. Do the benefits exceed the cost? Compare one agent with the smallest multi-agent design on the same models, tools, workload, and test set. Measure quality, coordination failures, median (p50) and p95 latency, cost, review time, unauthorized actions, duplicate actions, and privacy violations.

Test stale policy, timeouts, contradictory records, prompt injection, and partial action failure. Decide in advance what results would cause you to remove the extra agents. Remove them if they do not improve quality or reduce risk enough to justify their cost and p95 latency.

The goal is not the largest team. It is the smallest system that can do the work reliably.

References

1. Cemri, M., Pan, M. Z., Yang, S., et al. (2025). Why Do Multi-Agent LLM Systems Fail? NeurIPS 2025. Paper
2. Kim, Y., Gu, K., Park, C., et al. (2026). Towards a Science of Scaling Agent Systems. arXiv:2512.08296. Paper
3. Yang, Y., Qu, C., Wen, M., et al. (2026). Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity. arXiv:2602.03794. Paper
4. Zhang, Y., Liu, F., Shan, Y., et al. (2026). SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems. ACL 2026. Paper
5. Wunderlich, F. V., Kaesberg, L. B., Wahle, J. P., et al. (2026). Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling. ACL 2026 SRW. Paper
6. Alavi, K., Yeltay, Z., Flek, L., & Karimi, A. (2026). More Agents Improve Math Problem Solving but Adversarial Robustness Gap Persists. Findings of ACL 2026. Paper
7. Chen, N., Tong, Y., Yang, Y., et al. (2026). Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation. Findings of ACL 2026. Paper