When do you need a hierarchical agent team instead of a single supervisor?

You need a hierarchical agent team when one supervisor can no longer hold the whole system in a single routing decision, and the work splits into distinct domains that each deserve their own coordinator. Both conditions have to hold at once. If the workers do not cluster into clean domains, first rethink how you have divided the tasks before adding another manager.

What is a "supervisor of supervisors" in multi-agent systems?

A supervisor of supervisors is a top manager whose workers are themselves supervisors, each running its own pool of workers. A team can be implemented as a subgraph; individual worker nodes need not be subgraphs. The manager routes a request to the right domain's supervisor, which then routes to the right worker inside that domain.

How many agents before you should go hierarchical?

There is no hard agent-count threshold. Measure whether a flat supervisor still routes requests to the right worker as you add specialists. Add a tier only if routing degrades and the workers split into clear domains that each need their own coordinator.

What is the difference between supervisor and hierarchical multi-agent architectures?

With one supervisor, the same agent assigns work to all specialists. A hierarchical team adds team leads below a top-level manager, so choosing a worker can take two decisions before the task begins. The hierarchy can help a larger system send work to the right team, but adds model calls and waiting time.

How does state work across layers in a hierarchical agent system?

State flows down the tree as a request and back up as a result along the same path. The manager routes to a domain supervisor, which routes to a worker, and each layer sees the request with progressively narrower context. The manager never talks to a worker directly, which keeps each coordinator's context small but means a bad answer could have started at any layer.

What are the downsides of hierarchical agent teams?

The main downside is added coordination cost: every layer adds its own routing calls, latency, and places for a request to be misrouted. If one supervisor already chooses the right specialists reliably, adding another layer may increase cost without improving the result.

Hierarchical agent teams: how agents manage other agents

Hierarchical agent teams have a top-level manager, team leads, and specialist workers. The manager chooses a team, and its lead assigns the work. This can help when one supervisor struggles to choose among many specialists and the work falls into clear areas. Every extra layer adds model calls, waiting time, and another chance to send work to the wrong agent. If one supervisor still handles the team reliably, another layer may not help.

TL;DR A manager chooses a team, its lead chooses a worker, and results return through those same steps. This gives each manager fewer choices but adds calls and possible mistakes. Test it against one supervisor on your own tasks; having more agents does not automatically mean you need more managers.

What hierarchical agent teams are

A hierarchy has three kinds of node. At the top sits a manager whose only job is to decide which domain a request belongs to and hand it down. Below the manager sit supervisors, one per domain, and each supervisor decides which of its own workers should act. At the bottom are the workers that do the actual task. The manager never talks to a worker directly. It talks to supervisors, and supervisors talk to workers.

The structural idea is that each domain supervisor owns a working team, not just a single model call. The manager connects those teams. That nesting makes it a hierarchy rather than a wider flat team, and each routing layer adds cost.

How a hierarchy works

A request enters at the top. The manager picks a domain and passes the relevant request context to that domain's supervisor. The supervisor chooses a worker. The worker runs, and its result returns through the supervisor to the manager. Two routing decisions happen before the worker starts.

The mental model is a company org chart. A director does not do the analysis, they route it to the right team lead, who routes it to the right analyst. The value is that no single coordinator has to know every worker in the building. The cost is that every request pays for two managers before it reaches someone who produces output.

Because requests travel through more nodes than in a flat pattern, check the graph's step limit against real paths. Set a limit that allows legitimate work while still stopping runaway loops. LangGraph's recursion-limit guidance explains the mechanism; a fixed number is not right for every hierarchy.

What it looks like in code

Each domain supervisor delegates to its own specialists. The top manager delegates to those supervisors, not directly to the specialists. This sketch assumes the named worker tools wrap specialist agents and shows where the second routing tier lives.

from langchain.tools import tool
from langchain.agents import create_agent
 
research_supervisor = create_agent(
    model="anthropic:claude-sonnet-4-5",
    tools=[search_worker_tool, source_check_worker_tool],
)
writing_supervisor = create_agent(
    model="anthropic:claude-sonnet-4-5",
    tools=[draft_worker_tool, edit_worker_tool],
)
 
TEAMS = {"research": research_supervisor, "writing": writing_supervisor}
 
@tool
def delegate_to_team(domain: str, description: str) -> str:
    """Route a request through a domain supervisor."""
    result = TEAMS[domain].invoke(
        {"messages": [{"role": "user", "content": description}]}
    )
    return result["messages"][-1].content
 
manager = create_agent(
    model="anthropic:claude-sonnet-4-5",
    tools=[delegate_to_team],
    system_prompt="Choose a domain and delegate to its supervisor.",
)

Source: LangChain, subagents.

Figure: LangChain, multi-agent architectures, recolored to the page. This shows a flat supervisor for comparison, not the full hierarchy above.

When a hierarchy is the right call

The right call is a system whose flat supervisor routes poorly as specialists are added. Measure the routing decisions on representative requests rather than assuming a fixed agent-count threshold.

The second condition is that the work splits into domains that each need their own coordinator. If you can draw clean lines, say a research domain, a writing domain, and a data domain, where each has its own set of workers and its own logic for choosing among them, then a supervisor per domain earns its place. The manager routes between domains, and each supervisor owns the choices inside one.

Both conditions have to hold at once. Many workers without domain lines is a sign your task decomposition is off, not a reason to add a tier. A hierarchy makes sense only when the size is real and the seams are real.

When a hierarchy is overengineering

A manager above a single supervisor has only one destination, so it adds a call without making a useful domain choice. A manager above research and writing teams has two destinations even if the total worker count is small. The question is whether separating those decisions improves the result enough to justify the additional layer.

An extra layer adds a place where routing can go wrong. A manager can choose the wrong domain before a supervisor chooses a worker. Narrower domain choices may also improve accuracy, so compare end-to-end completion rather than assuming the additional tier helps or hurts.

If the flat supervisor still routes correctly, the extra tier has no demonstrated benefit. Test that simpler design before adding a manager.

What it costs

Each added tier introduces routing calls before a worker starts. A flat supervisor may route directly to a worker; a two-tier hierarchy first chooses a domain supervisor and then a worker. Compare those extra calls with any routing improvement. The cost depends on how often each tier is invoked.

The cost is not only tokens. Sequential routing calls add latency, and a wrong answer can originate at the manager, supervisor, or worker. Trace each decision so you can find the first wrong turn. On short tasks, routing may take more time than the work it coordinates.

What to trace

A hierarchy hides which layer went wrong, so trace each tier as its own decision, not the run as a whole:

Routing accuracy per layer. Whether the manager picked the right domain and the supervisor picked the right worker, sampled separately. A hierarchy that routes badly at the top wastes every call below it.
Misroute depth. When an answer is wrong, which layer first sent it off course. Without this you cannot tell a bad manager from a bad worker, and you will tune the wrong one.
Recursion-limit hits with the path. Record the last node, recent transitions, and task state. A hit alone cannot distinguish a valid deep run from a loop.
Coordination tax. Routing tokens and latency as a fraction of the whole run. When the tax rivals the work, the tier is not paying for itself.

What to do about it

Run the flat supervisor and a grouped version on the same representative requests. Record the chosen domain, chosen worker, final result, routing time, and model cost. Add the tier only if the grouped version improves completed work enough to pay for its extra routing.

If you keep the hierarchy, trace every handoff and set the graph's step limit from valid paths. When a run hits the limit, inspect its trace before raising the number. See the supervisor pattern for the flat baseline and the eight agent patterns for related designs.

Sources

LangChain: Multi-agent architectures