Skip to content

Instantly share code, notes, and snippets.

@ukslim
Last active March 4, 2024 09:57
Show Gist options
  • Save ukslim/0a52d8615c52b5c494fd772cd10a4802 to your computer and use it in GitHub Desktop.
Save ukslim/0a52d8615c52b5c494fd772cd10a4802 to your computer and use it in GitHub Desktop.
Avoiding Floating Point errors by scaling up.
// TL:DR
// In JS (and most other languages using floating point)
// 0.3 - 0.2 != 0.1
// 0.6 + 0.3 != 0.9
// These might cause problems for you.
// But!
// (0.3 * 10) - (0.2 * 10) == 0.1 * 10 == 1
// (0.6 * 10) + (0.3 * 10) == 0.9 * 10 == 9
// Use this to your advantage.
// For example, if your get inputs in numbers representing ££.pp, multiply by 100 and do your maths in pennies.
function assert(b: boolean) {
if(!b) { throw new Error('assert failed')};
}
// This is the idea I was originally investigating - the idea here is
// that we do purely string manipulation to move the decimal point `dp`
// positions to the right, ignore the remainder, and parse that as an integer.
// Hence we always work in integers and there is no opportunity for binary FP to mess with it
function parseRaised(s: string, dp: number): number {
const parts = s.split('.');
const integer = parts[0];
const fractional = (parts[1] || '').substring(0, dp).padEnd(dp,'0');
return Number(integer + fractional);
}
assert(parseRaised('100', 3) == 100000);
assert(parseRaised('200.2', 3) === 200200);
assert(parseRaised('300.21', 3) === 300210);
assert(parseRaised('400.213', 3) === 400213);
assert(parseRaised('400.213654', 3) === 400213);
assert(parseRaised('123456789.1234567', 3) === 123456789123);
// Now let's try various approaches. The idea is that all of these might be expected to
// print 0. Which ones do?
const inputs = [ '0.4', '0.3', '0.2', '0.1'];
// This doesn't return 0 because of the famous (0.3 - 0.2 != 0.1) problem.
// 0.3 and 0.2 are both approximations, and when subtracting the approximations compound
// into significant digits.
const basic = inputs
.map(s => Number(s)) // Turn the strings-representing-decimals to JS numbers
.reduce((sum, x) => sum - x, 1);
console.log('basic', basic); //
// I thought this would fail for the same reason as "basic", but in fact it works.
// This is because even though Number('0.2') only approximates 0.2, Number('0.2') * 10 is
// exactly 2. IEEE754 FP says that after an operation like multiplication, the result is rounded
// to the nearest "representable" number, 53 binary places (equivalent to around 16 decimal places)
// So, (0.2).toFixed(20) == '0.20000000000000001110',
// but also 10 * '0.20000000000000001110' == 2, exactly!
// Number.isInteger(10 * '0.20000000000000001110')
const withScaling = inputs
.map(s => Number(s) * 1000) // Turn them into JS numbers, but scale up by 1000
.reduce((sum, x) => sum - x, 1000);
console.log('alsoBasic', withScaling);
// Same as above. Not actually significant with these numbers, because all the numbers
// we trunc() are already integers for the reason given above
const truncated = inputs
.map(s => Math.trunc(Number(s) * 1000)) // Scale up by 1000 *and trunc* to integer
.reduce((sum, x) => sum - x, 1000);
console.log('truncated', truncated);
// And this is with my string manipulation function - but I think `withScaling` shows this is
// unnecessary
const withintegers = inputs
.map(s => parseRaised(s, 3)) // use our parseRaised, so at no point is a non-integer JS number involved
.reduce((sum, x) => sum - x, 1000);
console.log('withIntegers', withintegers);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment