GPT-5.6 Luna reportedly finds 75% of code review bugs for 3.6% of GPT-6 Astra cost

The benchmark pits GPT-5.6 Luna against GPT-6 Astra in a controlled code review task across 50 pull requests from five open-source projects. Luna’s output token pricing is 42x lower than Astra’s, driving a 28x cost difference per review. This cost advantage comes with a trade-off: Luna’s precision is 74%, meaning roughly one in four findings is incorrect, compared to Astra’s 96%. For teams already skeptical of AI-generated feedback, this noise could reduce trust in automated reviews.

Luna’s performance varies significantly by codebase and bug type. It matches Astra’s bug detection in Sentry, Discourse, and Grafana but lags in Cal.com and Keycloak, where it found only 6 of 14 verified bugs. The gap widens for security bugs, where Luna detects 9 of 24 compared to Astra’s 19. Keycloak’s identity and access management logic exposes Luna’s limitations, as it fails to catch issues like federated recovery code reuse or permission overrides that require multi-line reasoning.

Despite its shortcomings, Luna finds 25 bugs Astra misses, primarily in data, logic, and concurrency. This suggests the model is not uniformly inferior but has strengths in specific areas. The cost savings, $0.0030 per verified bug for Luna versus $0.061 for Astra, could justify its use for non-critical code, where false positives are tolerable. However, the precision gap means teams must either accept higher noise or invest in manual verification for Luna’s findings.

The benchmark’s methodology includes a verification step where both models judge findings anonymously, reducing bias but not eliminating it entirely. Astra’s role as a judge could slightly favor its own findings, though the 91% agreement rate between judges suggests this effect is minimal. The results highlight that AI model selection for code review is not binary; it depends on the balance between cost, precision, and the criticality of the code being reviewed.

For engineers, the takeaway is that cheaper models like Luna may be viable for routine correctness checks but require caution in security-sensitive or complex logic scenarios. The data does not support a one-size-fits-all recommendation, as the trade-offs are use-case dependent. Teams must define their tolerance for false positives and missed bugs before integrating such tools into their workflows.