Agent Build-Off — five stacks, one brief

The five arms who ran, on what, and each arm’s own caveat — press Play to open the built demo

What is not comparable read these before the scoreboard, not as footnotes

codex-astra is not a single-model run

Its ~18.6M-token total spans three models (gpt-6-astra, gpt-5.6-sol, codex-auto-review) across five delegated agents running partly in parallel. The other three arms each ran one model in one agent.

So its token total is not comparable to theirs at all: read its 98,882 output tokens as a sum across three models, and do not weigh its full total against claude-opus5’s 240,076 output tokens on the same axis.

The arms disagree on what the token totals count

Each arm decides differently whether the later session that wrote its stats file is inside its own token totals: codex-astra excludes it, opencode-qwen38 excludes it, opencode-qwen38-superpowers includes it, claude-qwen38 includes it, and claude-opus5 does not say.

Each arm’s own answer is printed on its card above, beside its token figures.

codex-astra reported in Portuguese

Its STATS.md is written in Portuguese; the other four are in English, with different section headings. This page’s own copy is English.

But every figure below is the arm’s own number, translated into the data, not re-measured.

The other limits

The three local arms cost nothing to run and took far longer. opencode-qwen38 took 90 minutes; claude-qwen38 144 minutes on a clock still running when its numbers were read; opencode-qwen38-superpowers spans an 11-hour raw clock (682.33 min, including a mid-task laptop sleep), of which 109.3 min — about 1 h 49 m — is active time. The two hosted arms finished in well under an hour but bill for their tokens.
opencode-qwen38 is the only arm without the superpowers skill library. Its opencode sibling invoked brainstorming, writing-plans, TDD and systematic-debugging; the operator states the two hosted arms had the library loaded too, and claude-qwen38’s transcript carries it, though that session invoked no skill through the Skill tool. The plain-versus-superpowers contrast therefore lives in the two opencode arms, not across the board.
Only codex-astra has a measured frame rate — 60.00 fps mean over a two-minute desktop sample. The other four record frame rate as not collected. That is a gap in their verification, not a win to credit to this arm.
hyper-runner-fable never ran and is absent from this comparison entirely.

The scoreboard every shared metric, sortable — click a header; gaps are uncollected metrics, not zeros

All metrics, all five arms

Every value is from bench/agent-build-off/results/arms.json. Click any column header to re-rank; double-click to reset. There is no green “best” highlight: more tokens or more lines is not “better,” and most columns are not like-for-like across rows (see the per-column notes below the table).

Arm	Wall (min)	Active (min)	Out tok	Uncached in	Cache-read	Reason tok	Prompts	Questions	Tool calls	Src lines	Src files	Doc lines	Direct deps	Pkgs	Bundle raw KB	Bundle gzip KB	Build (s)	Tests	Defects	FPS

Bases differ per row — the arms.json notes carry the detail; the load-bearing ones: Tokens — codex-astra’s are a three-model, five-agent sum (see the caveat above), not comparable to the single-model arms; the stats-writing session is inside some arms’ totals and not others. Wall time — opencode-qwen38 measures “first file to final edit,” the others “first prompt to final report”; opencode-qwen38-superpowers’s wall clock includes a long laptop sleep, so its Active column is the fair read. Tests — five different meanings: opencode-qwen38’s 9 are custom CDP browser assertions, opencode-qwen38-superpowers’s 65 are Vitest units, codex-astra’s 50 are Vitest + Playwright, and claude-opus5 used no unit-test framework (browser automation and a scripted autopilot instead) and claude-qwen38 used none at all, so those two cells are gaps, not zeros. Source lines — codex-astra reports tests separately; opencode-qwen38-superpowers folds a test file into its count. Bundle — codex-astra’s raw KB is the whole dist folder (assets included), not the JS/CSS-only figure the others report, and it published no gzip number.

A few dimensions, drawn read each caption for the caveat that the bars cannot carry

Time to build

Wall clock and active time, minutes, log scale. Active time is reported only where an arm split it out.

The two hosted arms finished fastest by the clock. opencode-qwen38-superpowers’s 682.33-minute wall clock is a raw span that includes a nine-hour laptop sleep; its 109.3-minute active bar is the honest build time. opencode-qwen38 and claude-opus5 report a single clock and no active split, so their Active bars are gaps.

Output tokens

Tokens the model generated. The most comparable token class across the arms — but read the caption.

codex-astra’s 98,882 is a sum across three models and five agents, not one model’s output; the other three bars are each a single model in a single agent. Cache-read input (millions of tokens) dwarfs output for every arm and lives in the scoreboard, not here.

Source lines shipped

Lines under src/ at the end of the session.

codex-astra’s 1,435 excludes its test lines, reported separately; opencode-qwen38-superpowers’s 2,136 folds a test file into the count. The bars are close, but not counted the same way.

Bundle size, gzipped

Gzipped production bundle, KB. Lower is a lighter first paint — not a quality verdict.

opencode-qwen38-superpowers ships the lightest bundle by hand-rolling physics; the two Rapier arms carry the WASM engine. codex-astra published no gzip figure, so its bar is a gap — its reported 5,836 KB is the whole dist folder with assets, a different measurement.