TECH Signal 595 2 feeds carried it
Study tests 26 verification technique prompts against coding agents on Rust Zstd implementation
↗
Illustration only Photo by H&CO on Unsplash
An evaluation compared 26 prompt conditions, including TDD, formal methods, fuzzing, and property-based testing, to see whether instructing coding agents to use specific verification techniques improves implementation correctness on a Rust Zstd task.
The study directly addresses whether the defaults developers use with coding agents are effective, and whether simple prompt addendums can meaningfully improve output quality. The provided material cuts off before presenting actual results, so the findings are not available in this extract.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The evaluation tested 26 prompt conditions including TDD, formal methods (Lean 4, Verus, TLA+), fuzzing, property-based testing, and a joke "Make no mistakes" condition, all on a Rust Zstd implementation.
The author pre-registered predictions that TDD would underperform, formal methods would not overperform good test techniques, and that the "Make no mistakes" instruction would not outperform no instructions at all.
Four agent skills were also tested, including the ECC Rust test skill and the Trail of Bits property test skill, with the author predicting none would outperform default behavior due to their size and lack of directly useful agent instructions.
THE CLUSTER