TECH Signal 466
AI painting model trains faster with pairwise judging and hand-rated reference pool
↗
A language model trained to paint by writing code improved after replacing nine reward signals with pairwise judging against a hand-rated reference pool, reaching its previous plateau three times faster and compressing code from 13,500 to under 2,000 tokens.
For engineers building RL systems on subjective tasks, this shows that reward design matters more than model or data. Pairwise comparison against a curated reference pool opens up dynamic range that absolute scoring compresses. The code compression also suggests the model learned more efficient solutions when reward signals were de-correlated.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The original nine-signal rubric plateaued at 0.65 reward because quality judges were correlated 0.85-0.95 and code length saturated by step thirty.
Replacing absolute scoring with pairwise judgment against 117 hand-rated love-tier examples from a pool of 1,664 images opened up the reward dynamic range.
The new four-component rubric reached the previous plateau three times faster and compressed generated code from 13,500 to under 2,000 tokens.
THE CLUSTER