Skip to content

Instantly share code, notes, and snippets.

@adamcameron
Created October 26, 2013 14:21
Show Gist options
  • Save adamcameron/7169987 to your computer and use it in GitHub Desktop.
Save adamcameron/7169987 to your computer and use it in GitHub Desktop.
Demonstrates FP inaccuracy
<cfscript>
// will result in floating point inaccuracy
all = precisionEvaluate((60 / 55) / (40 / 55));
a = precisionEvaluate(60 / 55);
b = precisionEvaluate(40 / 55);
c = precisionEvaluate(a / b);
writeDump(var={
all = all,
a = b,
b = b,
c = c
});
// will NOT result in floating point inaccuracy
mc = createObject("java", "java.math.MathContext");
mc = mc.DECIMAL128;
_60 = createObject("java", "java.math.BigDecimal").init("60", mc);
_55 = createObject("java", "java.math.BigDecimal").init("55", mc);
_40 = createObject("java", "java.math.BigDecimal").init("40", mc);
a = _60.divide(_55, mc);
b = _40.divide(_55, mc);
c = a.divide(b, mc);
writeDump(var={
all = [_60,_55,_40],
a = a,
b = b,
c = c
});
</cfscript>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment