AI Agents Carry Everything, Forget the Point
Agent swarms and sub-agents fail when they carry everything: every responsibility on the split, every bit of context loaded at once. The fix is structure, not a smarter model.
Agent swarms don't fail because the models are weak. They fail because the work was divided badly and the context was loaded all at once. The fix is structure: narrow, non-overlapping scope, and just-in-time context.
Give an agent a long, layered job and watch it forget its own instructions by the end. Give several agents jobs that overlap and watch them loop. These feel like two different failures. They're the same one. The agent was handed the whole thing at once, all the responsibility or all the context, and a system carrying everything it doesn't need does nothing well.
For years the fix people reach for is to give it more. A bigger context window. A smarter model. A supervisor to keep watch. All three treat the symptom, and all three sit downstream of the real problem, which is that the thing was built without structure. The model was never the bottleneck. The way the work was carved up, and the way the information was handed over, is where it breaks.
A swarm breaks at the split
Start with the split. When researchers at the University of California, Berkeley traced why multi-agent systems actually fail, they built a taxonomy from a couple hundred real agent runs. The largest chunk of failures, around forty-two percent, came from system design. Unclear prompts. Agents not sure what role they're playing. Tasks split too coarse or too fine to get reassembled into a working whole.
42%
The next largest, about a third, was inter-agent misalignment. Agents resetting their own conversation, withholding information, ignoring what another agent said. Only about a fifth was verification, the quality-check step at the end.
So the dominant failure class is upstream, in how the job got specified and divided before anything ran. Overlapping scope and no clear edge between one agent's work and the next are not grumbles. They're the root.
And it breaks over time
Now the half people miss. A single agent facing a long, layered task loses the plot even when the scope is clean. A classic finding, from Google Research, showed models use the information at the start and end of a long context better than the information buried in the middle. The details sitting at the middle of a long brief are the ones most likely to become invisible, even when they're technically still in the window.
A separate finding called it context rot. Across eighteen frontier models, output quality degrades as the input grows, and it starts early, long before the window fills. The benchmarks that look great, the needle-in-a-haystack tests, don't catch it because those only test whether a model can find one fact, not whether it can hold a whole task coherently.
This is where loading everything up top makes things worse instead of better. The agent isn't being given more clarity, it's being given a bigger pile to lose its place in. And when the pile is layered and the work goes long, the thing that's lost is usually the thing that mattered.
This one is from my own desk
I can tell you exactly what this looks like in practice. On content work I run long, multi-session pieces against a set of standing rules. A few rounds of feedback in, I noticed the original rules had quietly gone missing. The later feedback had crowded them out, so the system was now working off the recent notes and had forgotten the foundation they were built on. The model was fine. The information had been handed over once, up front, and then never surfaced again.
It's the same instinct twice. In the product I build, TikSense, the agents are scoped. Each track is a small, specialized job with its own prompt, configs, and algorithms, and no track carries the whole problem. And separately, when a workflow runs long, the system is built so that the right context gets loaded at the step that needs it, and gets re-loaded when a task is redone. The scope is about who owns what. The context is about what's in the head at the moment of action.
The fix is structure, and it's the same in both cases
So the answer isn't more context and it isn't a smarter model. It's deliberate structure, and it applies on both axes.
On scope you design the decomposition so each agent's job is narrow, specific, and mostly non-overlapping. You can do this statically, when you're building, or dynamically, at runtime, re-scoping as you learn. Either way you're aiming at the same end state: a defined, bounded job per agent, with a clean edge between them.
- 1Define the job narrowly, so one agent owns it end to end.
- 2Set a clean edge: what this agent owns, and where the next one starts.
- 3Avoid overlap; the loop is a scope bug, not a model bug.
On context you stop pre-loading everything and load what's needed at the point of action. The engineering guidance now has a name for this, just-in-time context, and it's concrete: keep light references around and pull the data in when the agent actually needs it, use progressive disclosure, and write things down. When the task has to be redone, the relevant knowledge gets loaded again, fresh, instead of using whatever degraded copy is still rattling around in the context.
There's a rule underneath both. An agent should only be holding the piece it's acting on. The piece of responsibility, and the piece of information.
Why this is expensive
And it is expensive when it's wrong. Agents don't just reason, they act. They make calls, move things, edit real work. When scope is blurred and context is stale, errors don't cancel out, they compound. One agent's slightly-off output becomes the next agent's ground truth, and the mistake travels down the line growing. It surfaces late, at the step that actually changes something, which is the step where recovery is most expensive. That's also why the failures feel like they come out of nowhere. The agent closes the loop, confident, and the missing detail only shows up when someone downstream notices.
A boundary you add after the structure
Once the scope and the context are right, a second question starts to matter. What is each agent allowed to decide on its own, and what does it escalate. I wrote about that separately, in the Agentic Authority Escalation Model. Scope and authority are different decisions. Scope is the architecture problem, what each agent owns. Authority is the boundary problem, what it can commit to without a person in the loop. You solve the first before the second. Most systems that spiral are a structure problem wearing an authority costume.
That also fits the way I think about enterprise AI in general, the argument that the model is infrastructure and the system around it is the product. The model was never going to save a system with no structure. The system you built around it was.
The structure is the product
If you take one thing, take this. An agent fails when it carries everything it doesn't need, all the responsibility on the split, all the decaying context on the time axis, and the fix is the same in both cases. Deliberate structure, so it's holding only the piece it's acting on. That's a design decision you make before anything runs, and it's why the skill worth building isn't picking models or wiring orchestrators. It's learning to see a workload the way a good project manager sees a team, and carving it into jobs that a barely-experienced agent, working on its own, can actually finish.
Which leaves the question I keep turning over. If the structure is the product, and the model is infrastructure, then the people who get good at this are the ones who stop asking which model and start asking what do I hand it, and when.