Benzi harness reports lower lines-read and cost-per-fix than Claude Code on 24-bug, 10-language benchmark

Benzi is a new entrant in the AI coding harness category, publishing benchmark claims against Claude Code and CodeGraph. The core claims are fewer source lines read, lower cost per fix, faster wall clock, and 78.2% on SWE-bench Verified. The article shows three charts (lines read, wall clock, cost) and four series (Benzi Sonnet, Benzi DeepSeek, Claude Code Sonnet, DeepSeek Harness DeepSeek). On lines read, Benzi Sonnet is lowest at 9,125 across 24 bugs, versus Claude Code Sonnet's 20,704 and DeepSeek Harness DeepSeek's 43,598; on cost at list price, Claude Code Sonnet climbs fastest with difficulty.

What adopting it costs in evaluation terms is accepting how the comparison was constructed. The 24 bugs are laid out easiest to hardest on an axis defined as Claude Code's turn count on that bug, which means the difficulty yardstick is the same harness the project is competing against rather than a neutral measure. The article also notes that on the wall-clock chart, Benzi's per-repo index build is not subtracted, so its seconds run higher than the 'warm figures quoted elsewhere on the page.' That caveat applies only to the wall-clock series, not to lines-read or cost, but it is the kind of methodology detail that disappears in a headline aggregate.

Where the comparison stops working is at the model-pairing boundary. The article explicitly says 'the two DeepSeek series run an order of magnitude cheaper than the two Sonnet ones,' and the totals bear this out: Benzi DeepSeek totals $2.66 and DeepSeek Harness DeepSeek totals $2.70 across all 24, versus $17.96 and $39.54 for the Sonnet pairings. So Benzi's cost edge is real against Claude Code Sonnet but is not a cost edge against the DeepSeek baseline, which the article itself frames as roughly twenty times cheaper per token. Two of the 24 cells in the cost table are blank, Benzi on Sonnet never solved http-parser and DeepSeek Harness DeepSeek never ran nats-server, so the 'all 24' totals hide two non-attempts that affect the slope fits used to argue Claude Code's cost climbs fastest.

Only one feed carries this event: the Hacker News Show HN post reproduced verbatim, with the linked page being Benzi's own benchmarks site. No independent review or replication feed has corroborated the figures, and the SWE-bench Verified 78.2% score is published only on the project's own page without configuration details in the extract. The article asserts 'every task, every attempt, verbatim, nothing held back,' but the logs themselves are not in the extract, so reproducibility is claimed rather than shown. For an engineer, the practical question is whether the harness, the 24-bug set, and the SWE-bench run configuration are available to rerun against their own workloads before treating these numbers as adoption guidance.