TECH Signal 382
Accurate byte-float color conversion uses 256 bins and floor, avoiding common rounding bias
↗
Illustration only Photo by Armand Khoury on Unsplash
Chris Lomont's note explains why the common divide-by-255 method for converting byte colors to floats is biased and proposes a 256-bin method with floor and clamping.
Image processing pipelines that convert between 8-bit and floating-point colors often use the naive divide-by-255 method, which biases values away from the endpoints. The 256-bin method preserves uniformity and roundtripping, so engineers should adopt it to avoid subtle color shifts in their applications.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The common method f = i/255.0 and round(f*255) gives non-uniform mapping, biasing away from 0 and 255.
The better method uses f = (i+0.5)/256.0 and floor(f*256) with clamping to [0,255] to ensure uniform bins.
The method ensures roundtripping: integer to float to integer returns the original value.
THE CLUSTER