TECH Signal 401
Aikido Security benchmark finds open-source DeepSeek V4 Pro outperforms closed models on CVE rediscovery across pooled runs
↗
Aikido Security tested 10 AI models on 32 fresh CVEs and found that three pooled runs of DeepSeek V4 Pro found 28 of 32 vulnerabilities, beating more expensive closed models at a fraction of the cost.
Open-source models can now match or exceed closed frontier models on vulnerability discovery when results are pooled across multiple runs, at roughly a quarter of the cost. The tradeoff is higher false-positive rates that the pipeline must filter, so the win is real but not free.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
DeepSeek V4 Pro found 28 of 32 vulnerabilities across three pooled runs, the highest recall of any model tested.
Three DeepSeek Flash runs cost $108 and matched Grok's best single pass, while three Pro runs cost $295 and outperformed Opus 5, Grok 4.6, and Sol.
Open-source models produced more false leads than closed models, making precision filtering a necessary part of the pipeline.
THE CLUSTER