Skip to content

Instantly share code, notes, and snippets.

@stuartpb stuartpb/comment.md
Created Aug 17, 2016

Embed
What would you like to do?
Original version of markusfisch/ShaderEditor#10#issuecomment-240450626

What that means is, while a mediump can contain a value up to 1024 without losing integral precision (the next increment it will be able to represent is 1026), it can only represent fractions of any granularity for much smaller ranges - from 513 to 1024, it'd be wholes, 256 to 512 it'd be halves, 128 to 256 quarters, 64 to 128 eighths, 32 to 16 the largest fractions would be sixteenths. Only values absolutely smaller than 8 are precise to the 1/32, smaller than 4 1/64, smaller than 2 1/128, and only fractions below a whole are precise to 1/256 (2^-8) - fractions of precision 2^-9 or 2^-10 only being under a half and a quarter, respectively (and fractions beyond that having no guarantees of precision per the spec). (Conversely, 1026 to 2048 will jump by twos, 2052 to 4096 by fours, 4104 to 8192 by eights, and 8208 to 16384 by sixteens.)

(FAKE EDIT: I just checked the OP, and what I'm about to describe doesn't quite match its figures, though that could just be because I'm bad at estimating framerates. I think my count of the powers might be off by a couple in one direction or another, either due to a misunderstanding of the spec and/or my phone's chip eschewing those aforementioned reserved bits and allocating them to fields closer to the IEEE sizes - but the underlying concept, that adequate precision is being lost beyond early spans in time, should be sound regardless of implementation, and may well be true on some other implementation out there.)

(ACTUAL EDIT: I definitely skipped the range from 16 to 8, so that's at least one jump I missed - I've updated the numbers below to reflect that, but not above because it would require more rewriting.)

(ACTUAL EDIT 2: I also missed the range from 32 to 64... ah heck, I'll rewrite it.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.