The Multi-Agent Scoreboard
Five builds. Was using a team of AIs worth it?
Over two weeks in late summer 2026 we built five programs with teams of AI agents: three arcade games, an app that turns recipe posts into a family cookbook, and a first-person shooter. Each time we asked the same question: did running several AIs at once beat one AI working alone? This page gives the answer for each build, shows what each one cost, and lists what we learned.
Five words you need
- Agent
- One AI session working on the code. Think of it as one teammate on a group project. A lead is the agent in charge; the others are subagents or lanes, each given one part of the job.
- Fan-out
- Splitting the work so several agents build different parts at the same time. The opposite is one agent doing everything in order.
- Contract
- A short file, written before the fan-out, that fixes the shapes everyone must agree on: what a function is called, what data looks like. Lanes build against the contract instead of talking to each other.
- Token
- The unit AI models read and write in; roughly three-quarters of a word. It is also the unit you pay in. Written tokens are what the model produced. Re-read tokens are the model reading its own conversation again before every reply, which it must do every single time.
- Rate limit
- A cap on how many tokens one account can use per session. Every agent on the same account draws from the same cap.
The answer
| Build | How the team was used | Did the team beat one agent? | Verdict |
|---|---|---|---|
| Brick Break | Every team pattern tried once, because the beginner's guide needed to show them | Never measured. Probably not. | Win, as a guide |
| Maze Muncher | Two leads at once, a review team, a build team, then three lanes making mazes | Never measured. | Mixed |
| Star Swarm | The biggest team: 18 subagent runs, two milestones built in parallel | Yes. Built in one night. | Win |
| Potluck | Five lanes sharing one rate limit, then a new agent sent after every bug | No. 17 hours where one agent needed 4 to 6. | Loss |
| Boom | One fan-out, watched closely, shut down after 68 minutes | No. The team wrote zero lines; one agent wrote all of it. | Win — for the rules, not the team |
In one sentence: the team beat one agent once (Star Swarm), lost once (Potluck), was never measured twice, and on Boom the lanes started but never wrote a line. Boom is the build Tommy calls our first real success — and it was built end to end by a single agent. What succeeded there was the rules: use the team once, cheaply, and put it down the moment it isn't paying off.
What each build cost
Here is the surprising thing about AI cost. Across all five builds the models wrote 2.4 million tokens. They re-read 680 million. That is because an AI has no memory between replies: before every answer it reads the whole conversation again, like a student who has to reread the entire textbook before answering each question. The longer the session, the bigger each reread. Re-reads are billed at a much lower rate than fresh text, but they are where the volume is — and every extra agent is another student rereading the same textbook. That is the real cost lever of the whole method.
| Build | Agents | Written | Re-read | Hours working | Messages from Tommy | Commits · tests |
|---|---|---|---|---|---|---|
| Brick Break | 1 lead + 12 subagent runs | 439K | 48M | 3.9 | 100 | 32 · 101 |
| Maze Muncher | 2 leads + 16 subagent runs | 381K | 101M | 5.2 | 183 | 56 · 249 |
| Star Swarm | 1 lead + 18 subagent runs | 448K | 190M | 4.5 | 134 | 79 · 327 |
| Potluck | 1 lead + 6 lane runs | 492K | 200M | 8.6* | 56 | 137 · 303 |
| Boom | 1 lead + 8 lane runs (wrote nothing) | 677K | 141M | 13.8 | 20 | 79 · 220 |
| All five | 6 leads, 49 subagent runs | 2.44M | 680M | 36 | 493 | 383 · 1,200 |
Build by build
Same four lines for every build. "One agent instead" is our estimate of what the same work would have cost with one agent in one session. The method: keep the lead's work and everything the lanes wrote, and drop what only existed because there was a team — teams that only reviewed, the briefings and reports, and each lane rereading the plan. Boom, whose lanes wrote nothing, is the closest thing we have to a control group.
Brick Break — a Breakout clone
- Team
- First one agent. Then subagents. Then a three-agent team that only reviewed the code and wrote a to-do list. Then a four-lane build team. Then two leads working in separate copies of the code. The beginner's guide had to show every pattern once, so it did.
- Cost
- 439K written, 48M re-read, about four hours of work across five days. The subagents wrote more than the lead did.
- One agent instead
- Our guess: 200–250K written and 25–30M re-read. A review-only team is pure overhead when one agent could just write the to-do list itself, and the whole game is about 1,300 lines.
- Verdict
- Win, as a guide. The point was the tutorial, and every pattern worked as described. As a build, a wash. The one failure that mattered had nothing to do with agents: the game showed up as a white box on Tommy's iPhone, and three rounds of fixes missed it because every test loaded the game's own page while Tommy was looking at the home page it was embedded in.
Maze Muncher — a Ms. Pac-Man-style game
- Team
- Two leads working at the same time, a review team, a four-lane build team, and finally three lanes working in parallel to add twenty new mazes, all built against a frozen maze-file format.
- Cost
- 381K written, 101M re-read, about five hours of work across three days, and 183 messages from Tommy — the most of any build.
- One agent instead
- Our guess: 200–250K written and 40–50M re-read. Two leads meant two agents rereading the same code.
- Verdict
- Mixed. The parallel maze build worked cleanly: no conflicts, 21 mazes, every one passing its checks. But the best bug found — ghosts whose eyes bounced forever instead of returning home, broken since level one — was found because twenty new mazes gave it twenty new places to show up, not because a reviewer spotted it. And nobody wrote down how long anything took, so we can't say whether the team saved time.
Star Swarm — a Galaga-style shooter
- Team
- One lead, a review team, a four-lane build team with the merge order decided up front, then five lanes in five separate copies of the code. The rule "every teammate gets its own copy" was invented here, after two agents sharing one copy kept undoing each other's work.
- Cost
- 448K written, 190M re-read, about four and a half hours of work — and the whole game, 79 commits and 318 tests, in a single 24-hour span.
- One agent instead
- Our guess: 250–300K written and 60–80M re-read, over two evenings instead of one.
- Verdict
- Win. The one build where the team clearly saved time. The price was rereading: the subagents did 69% of it, about two to three times what one agent would have read. Every bug was something only a human playing could find, like a rescue move that passed every automatic test but could never actually happen in a game.
Potluck — an AI recipe extractor
- Team
- The plan was five agents Tommy would drive by hand. He said no: "I do not want to try to control 5 agents." So one lead launched five lanes, each building against a contract file. That part worked. Then came bug-fixing, and a new agent got sent after every bug. Then, after Tommy's verdict, one agent finished everything.
- Cost
- 492K written, 200M re-read, plus a session that was lost. By its own notes: two hours of real work spread across 17 hours, 13 of them waiting on one shared rate limit. At one point 92% of a session's token allowance vanished in ten minutes.
- One agent instead
- The post-mortem worked this out itself: 3–4 hours of work, 4–6 hours start to finish, about a fifth of the tokens. In this page's units, roughly 300–350K written and 60–80M re-read.
- Verdict
- Loss, by the person paying. "I consider the multi-agent my first breakdown. I do not consider it a success." The app shipped and the family uses it every week, and the five lanes never once wrote conflicting code. What failed was the cost and the waiting, not the design.
Boom — a first-person shooter
- Team
- One lead following ten rules we published before writing any code. Fan out exactly once, four lanes, each writing its own tests. Both attempts started — eight lanes in all, each reading the contract and its part of the code — but none wrote a single file before being stopped. After 68 minutes the lead gave up on the team and built the rest alone, including four rounds of Tommy playtesting.
- Cost
- 677K written, 141M re-read, 13.8 hours of work in one session, and only 20 messages from Tommy. The eight lanes cost about 0.7M tokens altogether.
- One agent instead
- The same, minus 0.7M tokens and 68 minutes. Boom basically is the one-agent number: this is what a shooter with 220 tests and smart enemies costs when one agent builds all of it.
- Verdict
- Win, in Tommy's words. "I think Boom is our real first success with multi-agent. We did not overuse it, we used it like a scalpel." The lanes wrote nothing. What he's grading is the judgment around them: fan out once, where the plan said to; check the files on disk instead of believing an agent's status report; stop early and cheaply; spend the budget on playtesting instead. It produced the game people liked most.
- Revised
- Tommy, reading this back on 2026-09-08, once it was spelled out that no lane wrote a line: "so you are telling me that the entire Boom was single AI, not multi like we planned it… no wonder it felt right to me, it was back in my comfort zone." Worth keeping both halves. The verdict stands — the rules were the thing being graded, and they held. But the reason it felt like the best build was not that the team worked. It was that the team quietly stopped existing, and the build reverted to one agent working the way he already prefers to work. A method that feels good because it turned into your habit is not evidence the method works.
What we learned
- Use a team to build, never to fix. Once a human is testing and reporting bugs, the work is one thing at a time. Sending a fresh agent after each bug meant rereading 150–300K tokens to change a few lines.
- The contract is the real trick; the team is optional. Every time the shapes were frozen first, the lanes never collided. That habit makes a solo build better too.
- Rereading is what you pay for. 2.4 million written, 680 million re-read. Three agents that only review are three rereads of the whole project that add no code. Before adding an agent, ask what its rereading will cost.
- One rate limit turns a team into a line. Potluck's five lanes did two hours of work in seventeen because they were all waiting on the same cap.
- Check the disk, not the story. Agents report their own progress, and the report can be wrong: an agent that says it launched a teammate and didn't; a lane that looks busy and has written nothing. Look at the files. Look at the user's screen.
- "It felt right" is not evidence. Boom felt like the best build of the five, and it is — but the team wrote none of it. The fan-out failed and the build fell back to one agent, which is how Tommy already likes to work. When a method feels good, check whether it actually ran, or whether you simply returned to your comfort zone while its name stayed on the label.
- Write the rules before the code. Messages Tommy had to send per build: 100, 183, 134, 56, 20. Boom needed twenty messages in fourteen hours of work because its rules were written first.
If you remember one paragraph. Use a team of agents once per build, for big independent pieces, with the shared shapes frozen first, and only if they aren't sharing one rate limit. Expect to pay two to three times the rereading for the time you save. Once a person starts testing, go back to one agent — that is most of any real project. Watch the files, not the progress reports, and shut the team down the moment it isn't clearly winning. The agents were the headline of this series. The rules turned out to be the product, and Boom, where the rules ran the agents, is the build to copy.