Skip to content

Instantly share code, notes, and snippets.

@Flexi23
Created January 31, 2012 23:27
Show Gist options
  • Save Flexi23/1713774 to your computer and use it in GitHub Desktop.
Save Flexi23/1713774 to your computer and use it in GitHub Desktop.
GLSL 2D vector buffer in a texture with a custom floating point precision
/*
These are the helper functions to store and to restore a 2D vector with a custom 16 floating point precision in a texture.
The 16 bit are used as follows: 1 bit is for the sign, 4 bits are used for the exponent, the remaining 11 bit are for the mantissa.
The exponent bias is asymmetric so that the maximum representable number is 2047 (and bigger numbers will be cut)
the accuracy from 1024 - 2047 is one integer
512-1023 it's 1/2 int
256-511 it's 1/4 int and so forth...
between 0 and 1/16 the accuracy is the highest with 1/2048 (which makes 1/32768 the minimum representable number)
So this is a non-IEEE implementation (which would be a 5 bit exponent with a symmetric bias from 2^-15 to 2^16 and a 10 bit mantissa)
I hope anyone else knows a purpose for such a buffer and can use it too (in a fragment shader). ;)
Felix Woitzel, Jan/Feb 2012
(Twitter: @Flexi23)
attention: this is only tested on a AMD Radeon HD series chip so far and there might be oddities with Intel and Nvidia. I'll try and test it on other chips soon.
store: "gl_FragColor = encode2( v );"
restore: "vec2 v = decode2( texture2D( encoded_sampler, coord) );"
*/
vec2 encode(float v){
vec2 c = vec2(0.);
int signum = (v >= 0.) ? 128 : 0;
v = abs(v);
int exponent = 15;
float limit = 1024.; // considering the bias from 2^-5 to 2^10 (==1024)
for(int exp = 15; exp > 0; exp--){
if( v < limit){
limit /= 2.;
exponent--;
}
}
float rest;
if(exponent == 0){
rest = v / limit / 2.; // "subnormalize" implicite preceding 0.
}else{
rest = (v - limit)/limit; // normalize accordingly to implicite preceding 1.
}
int mantissa = int(rest * 2048.); // 2048 = 2^11 for the (split) 11 bit mantissa
int msb = mantissa / 256; // the most significant 3 bits go into the lower part of the first byte
int lsb = mantissa - msb * 256; // there go the other 8 bit of the lower significance
c.x = float(signum + exponent * 8 + msb) / 255.; // color normalization for texture2D
c.y = float(lsb) / 255.;
if(v >= 2048.){
c.y = 1.;
}
return c;
}
float decode(vec2 c){
float v = 0.;
int ix = int(c.x*255.); // 1st byte: 1 bit signum, 4 bits exponent, 3 bits mantissa (MSB)
int iy = int(c.y*255.); // 2nd byte: 8 bit mantissa (LSB)
int s = (c.x >= 0.5) ? 1 : -1;
ix = (s > 0) ? ix - 128 : ix; // remove the signum bit from exponent
int iexp = ix / 8; // cut off the last 3 bits of the mantissa to select the 4 exponent bits
int msb = ix - iexp * 8; // subtract the exponent bits to select the 3 most significant bits of the mantissa
int norm = (iexp == 0) ? 0 : 2048; // distinguish between normalized and subnormalized numbers
int mantissa = norm + msb * 256 + iy; // implicite preceding 1 or 0 added here
norm = (iexp == 0) ? 1 : 0; // normalization toggle
float exponent = pow( 2., float(iexp + norm) - 16.); // -5 for the the exponent bias from 2^-5 to 2^10 plus another -11 for the normalized 12 bit mantissa
v = float( s * mantissa ) * exponent;
return v;
}
vec4 encode2(vec2 v){
return vec4( encode(v.x), encode(v.y) );
}
vec2 decode2(vec4 c){
return vec2( decode(c.rg), decode(c.ba) );
}
@phtaylor
Copy link

Hi, this is exactly what I've been looking for. My math foo isn't strong enough to write something like this myself.
I am encoding an object id and distance into an RGBA buffer.. so 16 bits for the id, and 16 bits for the distance.

Thanks for sharing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment