{
  "campaign": "agent-build-off",
  "description": "Five coding-agent stacks, the same verbatim brief, each building the same 3D game from an empty folder in one session, in parallel, on 2026-09-07 (2026-09-06 BRT evening for codex-astra; claude-qwen38 ran later the same day, after the first four). Source vendored under demos/<arm>; per-arm self-reports under results/<arm>.md; this file is the single source of numbers for the dashboard.",
  "brief": "You are going to create a complete and impressive 3D demo runner game, using modules and libraries to reduce the need of new code. You should search the web for most updated information, stack and best practices. Select the stack based on easiness to implement a functional demo on reduced time. Question me for decisions and ambiguities while creating the plan.\n\n- SAVE a detailed SPEC AND PLAN to SPEC.md and PLAN.md on the folder before building\n- Create an AGENTS.md explaining the repo objective, poiting to both PLAN and SPEC files",
  "arms": [
    {
      "id": "opencode-qwen38",
      "harness": "opencode",
      "model": "qwen3.8-flash-next",
      "hosted": false,
      "label": "opencode + Qwen3.8 Flash Next (local)",
      "color": "#2a9d8f",
      "sourceFolder": "~/LocalProjects/hyper-runner",
      "statsSessionIncluded": false,
      "statsSessionNote": "Its own STATS.md states: \"Figures exclude the two later prompts that wrote this file.\"",
      "multiModel": false,
      "skills": [],
      "config": {
        "harness": "opencode",
        "harnessVersion": null,
        "modelId": "ddalcu Qwen3.8 Flash Next, MLX mixed 4/8-bit",
        "runtime": "served locally on the rig",
        "effort": null,
        "source": "bench/agent-build-off/results/opencode-qwen38.md"
      },
      "short": "Qwen3.8"
    },
    {
      "id": "opencode-qwen38-superpowers",
      "harness": "opencode",
      "model": "qwen3.8-flash-next",
      "hosted": false,
      "label": "opencode + Qwen3.8 Flash Next + superpowers (local)",
      "color": "#52b788",
      "sourceFolder": "~/LocalProjects/hyper-runner-superpowers",
      "statsSessionIncluded": true,
      "statsSessionNote": "Its own STATS.md states: \"Totals were read while this stats request was running, so they include it.\"",
      "multiModel": false,
      "skills": [
        "superpowers"
      ],
      "config": {
        "harness": "opencode",
        "harnessVersion": null,
        "modelId": "ddalcu Qwen3.8 Flash Next, MLX mixed 4/8-bit",
        "runtime": "served locally on the rig",
        "effort": null,
        "source": "bench/agent-build-off/results/opencode-qwen38-superpowers.md"
      },
      "short": "Qwen3.8 + superpowers"
    },
    {
      "id": "claude-opus5",
      "harness": "claude",
      "model": "claude-opus-5",
      "hosted": true,
      "label": "Claude + Opus 5",
      "color": "#e76f51",
      "sourceFolder": "~/LocalProjects/hyper-runner-claude",
      "statsSessionIncluded": null,
      "statsSessionNote": "not stated",
      "multiModel": false,
      "skills": [
        "superpowers"
      ],
      "config": {
        "harness": "Claude Code",
        "harnessVersion": null,
        "modelId": "claude-opus-5",
        "runtime": null,
        "effort": null,
        "source": "bench/agent-build-off/results/claude-opus5.md"
      },
      "short": "Opus 5"
    },
    {
      "id": "codex-astra",
      "harness": "codex",
      "model": "gpt-6-astra",
      "hosted": true,
      "label": "Codex + gpt-6-astra (multi-model)",
      "color": "#f4a261",
      "sourceFolder": "~/LocalProjects/hyper-runner-astra",
      "statsSessionIncluded": false,
      "statsSessionNote": "Its own STATS.md states plainly it excludes the session that wrote the file (\"O relatório não inclui os tokens usados para criar este arquivo.\"). A verification pass against the Codex session logs found the excluded-session explanation accounts for the direction of a self-report/log gap but not its full size (see results/codex-astra.md).",
      "multiModel": true,
      "skills": [
        "superpowers"
      ],
      "multiModelNote": "This arm's ~18.6M-token total spans THREE models (gpt-6-astra, gpt-5.6-sol, codex-auto-review) across FIVE delegated agents running partly in parallel. The other three arms each ran a single model in a single agent. codex-astra's token total is therefore NOT comparable to the other three arms' totals — this is a larger caveat than the statsSessionIncluded convention above.",
      "config": {
        "harness": "Codex",
        "harnessVersion": null,
        "modelId": "gpt-6-astra, with gpt-5.6-sol and codex-auto-review",
        "runtime": null,
        "effort": null,
        "source": "bench/agent-build-off/results/codex-astra.md"
      },
      "short": "gpt-6-astra"
    },
    {
      "id": "claude-qwen38",
      "harness": "claude",
      "model": "qwen3.8-flash-next",
      "hosted": false,
      "label": "Claude Code + Qwen3.8 Flash Next (local)",
      "color": "#146b5b",
      "sourceFolder": "~/LocalProjects/hyper-runner-qwen-cc",
      "statsSessionIncluded": true,
      "statsSessionNote": "Its own STATS.md counts the stats request as one of its six user prompts and runs its wall clock to the extraction moment, so the session that wrote the file is inside its own totals.",
      "multiModel": false,
      "skills": [
        "superpowers"
      ],
      "config": {
        "harness": "Claude Code",
        "harnessVersion": null,
        "modelId": "ddalcu-Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit",
        "runtime": "served locally on the rig, MLX proxy, variant xhigh",
        "effort": "xhigh",
        "source": "bench/agent-build-off/results/claude-qwen38.md"
      },
      "short": "Qwen3.8 on Claude Code"
    }
  ],
  "results": [
    {
      "arm": "opencode-qwen38",
      "metric": "wallMinutes",
      "value": 90,
      "note": "\"first files written to final source edit\" (00:15:44Z–01:46:00Z) — a different basis than claude-opus5/superpowers, which both measure first-prompt-to-final-report."
    },
    {
      "arm": "opencode-qwen38",
      "metric": "activeMinutes",
      "value": null,
      "note": "not distinguished from wall time; this arm reports only the one 90-minute clock"
    },
    {
      "arm": "opencode-qwen38",
      "metric": "tokensOutput",
      "value": 112802
    },
    {
      "arm": "opencode-qwen38",
      "metric": "tokensInputUncached",
      "value": 1268186
    },
    {
      "arm": "opencode-qwen38",
      "metric": "tokensCacheRead",
      "value": 4790430
    },
    {
      "arm": "opencode-qwen38",
      "metric": "tokensReasoning",
      "value": 0
    },
    {
      "arm": "opencode-qwen38",
      "metric": "userPrompts",
      "value": 10
    },
    {
      "arm": "opencode-qwen38",
      "metric": "questionsAsked",
      "value": 3
    },
    {
      "arm": "opencode-qwen38",
      "metric": "toolCalls",
      "value": 142
    },
    {
      "arm": "opencode-qwen38",
      "metric": "sourceLines",
      "value": 1170,
      "note": "src/ TypeScript + CSS"
    },
    {
      "arm": "opencode-qwen38",
      "metric": "sourceFiles",
      "value": 13
    },
    {
      "arm": "opencode-qwen38",
      "metric": "docLines",
      "value": 159
    },
    {
      "arm": "opencode-qwen38",
      "metric": "directDeps",
      "value": 17,
      "note": "10 runtime + 7 dev"
    },
    {
      "arm": "opencode-qwen38",
      "metric": "installedPackages",
      "value": 136
    },
    {
      "arm": "opencode-qwen38",
      "metric": "bundleRawKB",
      "value": 3463.2,
      "note": "index.js 3,460 kB + index.css 2.6 kB + index.html 0.6 kB"
    },
    {
      "arm": "opencode-qwen38",
      "metric": "bundleGzipKB",
      "value": 1182.4,
      "note": "index.js 1,181 kB + index.css 1.0 kB + index.html 0.4 kB"
    },
    {
      "arm": "opencode-qwen38",
      "metric": "buildSeconds",
      "value": 0.33
    },
    {
      "arm": "opencode-qwen38",
      "metric": "testsPassing",
      "value": 9,
      "note": "custom CDP browser-automation assertions (6 regression + 3 deterministic-jump), not a unit test framework — this arm has no Vitest suite"
    },
    {
      "arm": "opencode-qwen38",
      "metric": "defectsFound",
      "value": 8
    },
    {
      "arm": "opencode-qwen38",
      "metric": "meanFps",
      "value": null,
      "note": "not collected"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "wallMinutes",
      "value": 682.33,
      "note": "raw clock, first prompt to final report (02:12:49Z→13:35:09Z), including a 9h13m mid-task laptop sleep"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "activeMinutes",
      "value": 109.3,
      "note": "raw clock minus the sleep and minus 19m54s summed input-wait gaps"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "tokensOutput",
      "value": 285431
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "tokensInputUncached",
      "value": 875625
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "tokensCacheRead",
      "value": 13581216
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "tokensReasoning",
      "value": null,
      "note": "not stated; this arm's Tokens table has no reasoning row"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "userPrompts",
      "value": 10
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "questionsAsked",
      "value": 8
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "toolCalls",
      "value": 230,
      "note": "plus 1 malformed call not counted"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "sourceLines",
      "value": 2136,
      "note": "1,978 TS/TSX + 158 CSS; includes at least one test file (game/runtime.test.ts, 289 lines) inside the src/ count — a different convention than codex-astra, which reports test lines separately"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "sourceFiles",
      "value": 34
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "docLines",
      "value": 3201,
      "note": "SPEC 219 + PLAN 2,953 + AGENTS 29"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "directDeps",
      "value": 16,
      "note": "7 runtime + 9 dev"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "installedPackages",
      "value": 156
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "bundleRawKB",
      "value": 1177.34,
      "note": "index.js 1,175 kB + index.css 2,335 B"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "bundleGzipKB",
      "value": 319.96,
      "note": "index.js 319 kB + index.css 958 B"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "buildSeconds",
      "value": 0.155,
      "note": "reported range 0.147–0.162 s over three runs; mean shown"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "testsPassing",
      "value": 65,
      "note": "Vitest unit tests (vitest + jsdom), all passing — this arm's only verification method; a different basis than opencode-qwen38's 9 custom CDP browser-automation assertions, claude-opus5's null (no unit test framework, browser-automation/autopilot only), or codex-astra's 50 (Vitest + Playwright)"
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "defectsFound",
      "value": 6
    },
    {
      "arm": "opencode-qwen38-superpowers",
      "metric": "meanFps",
      "value": null,
      "note": "not collected"
    },
    {
      "arm": "claude-opus5",
      "metric": "wallMinutes",
      "value": 41.9,
      "note": "first prompt to final report (00:52:37Z–01:34:31Z)"
    },
    {
      "arm": "claude-opus5",
      "metric": "activeMinutes",
      "value": null,
      "note": "not distinguished from wall time; this arm reports only the one 41.9-minute clock, no sleep or wait deduction"
    },
    {
      "arm": "claude-opus5",
      "metric": "tokensOutput",
      "value": 240076
    },
    {
      "arm": "claude-opus5",
      "metric": "tokensInputUncached",
      "value": 374
    },
    {
      "arm": "claude-opus5",
      "metric": "tokensCacheRead",
      "value": 25978321
    },
    {
      "arm": "claude-opus5",
      "metric": "tokensReasoning",
      "value": null,
      "note": "not stated; this arm's Tokens table has no reasoning row"
    },
    {
      "arm": "claude-opus5",
      "metric": "userPrompts",
      "value": 3
    },
    {
      "arm": "claude-opus5",
      "metric": "questionsAsked",
      "value": 8
    },
    {
      "arm": "claude-opus5",
      "metric": "toolCalls",
      "value": 107
    },
    {
      "arm": "claude-opus5",
      "metric": "sourceLines",
      "value": 2111,
      "note": "src/ TypeScript + CSS"
    },
    {
      "arm": "claude-opus5",
      "metric": "sourceFiles",
      "value": 26
    },
    {
      "arm": "claude-opus5",
      "metric": "docLines",
      "value": 319
    },
    {
      "arm": "claude-opus5",
      "metric": "directDeps",
      "value": 15
    },
    {
      "arm": "claude-opus5",
      "metric": "installedPackages",
      "value": 74
    },
    {
      "arm": "claude-opus5",
      "metric": "bundleRawKB",
      "value": 3520.9,
      "note": "index.js 1,257 kB + Debris.js (Rapier, lazy) 2,258 kB + index.css 3.2 kB + rapier.js 2.7 kB"
    },
    {
      "arm": "claude-opus5",
      "metric": "bundleGzipKB",
      "value": 1199.5,
      "note": "index.js 347 kB + Debris.js 850 kB + index.css 1.3 kB + rapier.js 1.2 kB"
    },
    {
      "arm": "claude-opus5",
      "metric": "buildSeconds",
      "value": 0.44,
      "note": "reported range 0.39–0.49 s over three runs; mean shown"
    },
    {
      "arm": "claude-opus5",
      "metric": "testsPassing",
      "value": null,
      "note": "no unit test framework used; verification ran through 37 browser-automation calls and a scripted autopilot instead, not counted as tests"
    },
    {
      "arm": "claude-opus5",
      "metric": "defectsFound",
      "value": 5
    },
    {
      "arm": "claude-opus5",
      "metric": "meanFps",
      "value": null,
      "note": "not collected"
    },
    {
      "arm": "codex-astra",
      "metric": "wallMinutes",
      "value": 67,
      "note": "\"duração observada\", 1h07min (22:24–23:31 BRT), from the first to the last token-usage record; five delegated agents ran partly in parallel, so this is not the sum of each agent's own time"
    },
    {
      "arm": "codex-astra",
      "metric": "activeMinutes",
      "value": null,
      "note": "not distinguished from wall time; no sleep/input-wait split reported"
    },
    {
      "arm": "codex-astra",
      "metric": "tokensOutput",
      "value": 98882,
      "note": "sum across all three models (gpt-6-astra 54,035 + gpt-5.6-sol 42,146 + codex-auto-review 2,701); see arm's multiModelNote — not comparable to the single-model arms"
    },
    {
      "arm": "codex-astra",
      "metric": "tokensInputUncached",
      "value": 647559,
      "note": "stated as one total across all three models, not broken down per model"
    },
    {
      "arm": "codex-astra",
      "metric": "tokensCacheRead",
      "value": 17861000,
      "note": "arm's stated Total row (17.861 M); per-model rows are gpt-6-astra 11.114M + gpt-5.6-sol 6.040M + codex-auto-review 707,328"
    },
    {
      "arm": "codex-astra",
      "metric": "tokensReasoning",
      "value": 18106,
      "note": "already included inside tokensOutput's total (98,882); not additive"
    },
    {
      "arm": "codex-astra",
      "metric": "userPrompts",
      "value": null,
      "note": "not stated in this arm's own STATS.md; the log-verification addendum counts 53 user prompts across all 13 matched rollout files, but that scope includes delegated-agent turns and the stats-writing session, so it is not a like-for-like substitute"
    },
    {
      "arm": "codex-astra",
      "metric": "questionsAsked",
      "value": 3,
      "note": "\"3 perguntas\" from the 187-call tool taxonomy; the report does not state whether these were addressed to the user or exchanged between delegated agents"
    },
    {
      "arm": "codex-astra",
      "metric": "toolCalls",
      "value": 187,
      "note": "156 commands + 16 inter-agent messages + 5 delegations + 4 resumes + 3 questions + 2 waits + 1 agent-list query"
    },
    {
      "arm": "codex-astra",
      "metric": "sourceLines",
      "value": 1435,
      "note": "src/ only, excludes 536 lines of tests (reported separately); other arms (e.g. superpowers) may include test files inside their source-line count, so this is not strictly apples-to-apples"
    },
    {
      "arm": "codex-astra",
      "metric": "sourceFiles",
      "value": 16,
      "note": "excludes 5 test files (reported separately)"
    },
    {
      "arm": "codex-astra",
      "metric": "docLines",
      "value": null,
      "note": "not stated in this arm's STATS.md"
    },
    {
      "arm": "codex-astra",
      "metric": "directDeps",
      "value": 6,
      "note": "2 runtime + 4 dev"
    },
    {
      "arm": "codex-astra",
      "metric": "installedPackages",
      "value": 90
    },
    {
      "arm": "codex-astra",
      "metric": "bundleRawKB",
      "value": 5836,
      "note": "\"5,7 MB, 180 arquivos\" — total production dist folder (180 files), including the 2.2 MB of local assets; not decomposed into per-chunk raw/gzip like the other arms' figures, so not directly comparable to their JS/CSS-only totals"
    },
    {
      "arm": "codex-astra",
      "metric": "bundleGzipKB",
      "value": null,
      "note": "not measured; the arm's report gives only total dist size, no gzip breakdown"
    },
    {
      "arm": "codex-astra",
      "metric": "buildSeconds",
      "value": null,
      "note": "not recorded; the report states only that the production build passed"
    },
    {
      "arm": "codex-astra",
      "metric": "testsPassing",
      "value": 50,
      "note": "36 Vitest tests + 14 Playwright browser tests, all passing"
    },
    {
      "arm": "codex-astra",
      "metric": "defectsFound",
      "value": null,
      "note": "this arm's STATS.md has no \"defects found\" section, unlike the other three arms (see roster heading-gap note in results/codex-astra.md)"
    },
    {
      "arm": "codex-astra",
      "metric": "meanFps",
      "value": 60.0,
      "note": "mean over a 125.076 s desktop sample, Chromium 153, 1440×900, Apple M5 Pro through ANGLE Metal; codex-astra is the only one of the four arms with a measured frame rate — a gap in the other three arms' verification, not a win to credit to this one"
    },
    {
      "arm": "claude-qwen38",
      "metric": "wallMinutes",
      "value": 144,
      "note": "14:13:06–16:37 UTC, first prompt to extraction, ≈ 2 h 24 min. The report calls the session \"still open at extraction\" and its own compaction table lists rollovers at 16:46 and 17:08 UTC, after the end it quotes, so this span is a snapshot and includes the stats investigation from 16:28 on."
    },
    {
      "arm": "claude-qwen38",
      "metric": "activeMinutes",
      "value": null,
      "note": "not separated; the report gives only a phase split it calls rough — planning and research 15 min, documentation 1 min, code, builds and fixes ≈ 70 min"
    },
    {
      "arm": "claude-qwen38",
      "metric": "tokensOutput",
      "value": 149235,
      "note": "134,933 main session + 14,302 planning subagent; deduplicated by message id, keeping the maximum usage seen per id"
    },
    {
      "arm": "claude-qwen38",
      "metric": "tokensInputUncached",
      "value": 3538572,
      "note": "3,524,225 main + 14,347 subagent"
    },
    {
      "arm": "claude-qwen38",
      "metric": "tokensCacheRead",
      "value": 3194985,
      "note": "3,188,860 main + 6,125 subagent; cache creation is 0 because the local proxy never reports it"
    },
    {
      "arm": "claude-qwen38",
      "metric": "tokensReasoning",
      "value": null,
      "note": "not stated; this arm's Tokens table has no reasoning row"
    },
    {
      "arm": "claude-qwen38",
      "metric": "userPrompts",
      "value": 6,
      "note": "includes the prompt that requested the stats file"
    },
    {
      "arm": "claude-qwen38",
      "metric": "questionsAsked",
      "value": 4,
      "note": "one AskUserQuestion round of four questions, all answered with the recommended option"
    },
    {
      "arm": "claude-qwen38",
      "metric": "toolCalls",
      "value": 127,
      "note": "124 main session + 3 in the planning subagent, deduplicated by call id"
    },
    {
      "arm": "claude-qwen38",
      "metric": "sourceLines",
      "value": 2135,
      "note": "src/*.ts 1,824 + hud.css 246 + index.html 65"
    },
    {
      "arm": "claude-qwen38",
      "metric": "sourceFiles",
      "value": 19
    },
    {
      "arm": "claude-qwen38",
      "metric": "docLines",
      "value": 343,
      "note": "SPEC.md 181, PLAN.md 90, AGENTS.md 72"
    },
    {
      "arm": "claude-qwen38",
      "metric": "directDeps",
      "value": 3,
      "note": "1 runtime + 2 dev in package.json; the arm's STATS.md says \"3 runtime + 2 dev\", which its own Libraries table does not support"
    },
    {
      "arm": "claude-qwen38",
      "metric": "installedPackages",
      "value": 15,
      "note": "packages under node_modules on the rig; the arm's STATS.md says 20, and package-lock.json lists 66 entries, most of them platform binaries that are never installed"
    },
    {
      "arm": "claude-qwen38",
      "metric": "bundleRawKB",
      "value": 6868.46,
      "note": "dist/assets/index-*.js; the build warns about chunk size because all of @babylonjs/core is imported"
    },
    {
      "arm": "claude-qwen38",
      "metric": "bundleGzipKB",
      "value": 1516.54
    },
    {
      "arm": "claude-qwen38",
      "metric": "buildSeconds",
      "value": 35.5
    },
    {
      "arm": "claude-qwen38",
      "metric": "testsPassing",
      "value": null,
      "note": "no test framework was used; five plan items marked [browser] were never executed"
    },
    {
      "arm": "claude-qwen38",
      "metric": "defectsFound",
      "value": 7,
      "note": "found during the arm's own verification and fixed, including three that a type check alone would not have caught"
    },
    {
      "arm": "claude-qwen38",
      "metric": "meanFps",
      "value": null,
      "note": "not collected; the report lists sustained 60 fps under Not verified"
    }
  ],
  "configNote": "Run configuration for each arm, transcribed from that arm's own report named in `source`. A field the report does not state is null, never guessed; none of these reports records a reasoning effort or thinking level.",
  "shortNote": "A short display name for each arm, for chart rows and other narrow places. It names the same arm as `label`; it measures nothing.",
  "skillsNote": "`skills` lists the skill libraries loaded in that arm's session. Four of the five ran with superpowers; opencode-qwen38 is the arm without it. For the first four the operator stated it on 2026-09-08 and the per-arm reports do not record it; for claude-qwen38 the session transcript carries the library, though the build session invoked no skill through the Skill tool."
}