Verdict
The incumbent, ddalcu mixed-4/8 on mlx-serve 26.9.2, is the most responsive Flash-Next daily driver on the M4 Max 128 GB. Its T_turn (median of 3 reps) is 11.03 s at 32K and 12.35 s at 128K. That is 18% ahead of oQ4e on oMLX 0.7.0.dev2 (13.48 / 15.03 s).
The lead comes from TTFT. At 128K the warm turn responds in 2.1 s against 4.8 s, and the first turn in 178 s against 259 s. Decode ties at ~50 tok/s.
MTPLX 2.11.2 leads decode at 32K, but it refuses 128K and 256K on this Mac (HTTP 507, fit of 114,688 tokens). oMLX 0.6.4 trails 0.7.0.dev2 on the same weights. Agent quality (Terminal-Bench) was not measured.
Scoreboard
T_turn = tool_turn TTFT + 512 / median decode of the served warm scenarios (identical, append, tool_turn). It is the wait the user feels per turn. Wired peak is the highest wired_peak_gb across the measured bands. A failed gate eliminates the candidate (hit ≥0.90 at 32K/128K, no HTTP errors, swap ≤0.5 GB, correct answer). Wired above 102 GB is a warning in its own column, not an elimination. It is the normal operating point of mlx-serve (KV + a 16 GB prefix cache held in wired memory) when swap does not grow. Default sort is T_turn @32K, lowest first. Click a header to re-sort. Lower is better for T_turn, TTFT and wired; higher is better for decode. The best visible cell in each column is green, using that column's direction.
| Candidate | T_turn @32K (s) | T_turn @128K (s) | cold TTFT @128K (s) | decode @128K (tok/s) | wired peak (GB) | failed gates | warnings |
|---|
³ median of 3 reps (Stage B); other values: 1 rep (Stage A).
The 8K smoke run is left out of the charts because MTPLX telemetry is inconsistent at 8K. See etapa0-smoke.md.
T_turn vs context
T_turn = tool_turn TTFT + 512 / median decode of the served warm scenarios (identical, append, tool_turn), per context band. A gap in a line means the band was not measured.
Warm TTFT (tool_turn) vs context
Time to first token in the tool-turn scenario, with the cache already warm.
Cold TTFT vs context
Time to first token with an empty cache (first load of the prompt at that band).
Decode vs context
Generation speed (tok/s), median of the warm scenarios.
Cache hit by scenario
Share of tokens reused from the prefix cache, per scenario and candidate, band by band. Green = ≥0.90 (gate passes); red = below; gray = not measured.
Wired memory by context
Peak resident unified memory (wired_peak_gb) per context band.
512K capacity probe
Does each candidate reach the 512K band and hold a follow-up without running out of memory? "not tested" means the probe did not run for that candidate.
| Candidate | Reaches 512K | Follow-up ok | decode (tok/s) | cold TTFT (s) | Note |
|---|
Gaps and caveats
- MTPLX cache telemetry is not reliable at 8K.
- The verification hash covers only the final answer, not the full reasoning chain.
- oMLX does not implement YaRN, so its practical context ceiling is 262K.