Skip to content

Instantly share code, notes, and snippets.

@nothings
Created October 2, 2014 08:37
Show Gist options
  • Save nothings/06ac1d8ea61635d09e0f to your computer and use it in GitHub Desktop.
Save nothings/06ac1d8ea61635d09e0f to your computer and use it in GitHub Desktop.
ryg on luajit string-tofloat
Thinking about this some more: the decimal expansion of the halfway point between two numbers is always going to be either
...xyz000000
or
...xyz500000
We don't want to deal with the 5s if we can avoid it. Say the longest decimal expansion of a binary float is N digits (it's somewhere around 770, haven't calculated it exactly). Then the halfway point is gonna be all zeros after N+1 decimal digits, guaranteed.
To round, we just need to know if we're *exactly* on the edge between two numbers (in which case we might need to employ a tie breaker like round to nearest even) or slightly higher - in effect, is the number "1234.50000" or "1234.50000+"?
So we read at least N+1 digits exactly (LuaJIT just reads up to 800, which is definitely enough for doubles); beyond that, we just need to know if any of the following digits is nonzero (we have the '+') or not.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment