Skip to content

Instantly share code, notes, and snippets.

@sentientmonkey
Created April 1, 2010 22:51
Show Gist options
  • Save sentientmonkey/352480 to your computer and use it in GitHub Desktop.
Save sentientmonkey/352480 to your computer and use it in GitHub Desktop.
#include <math.h>
#include <stdio.h>
int main() {
double a = 16.99;
double b = 100.0;
printf("a is %f\n", a);
printf("b is %f\n", b);
printf("a*b is %f\n", a*b);
printf("floor(a*b) is %f\n", floor(a*b));
return 0;
}
a is 16.990000
b is 100.000000
a*b is 1699.000000
floor(a*b) is 1698.000000
@sentientmonkey
Copy link
Author

Oh yeah, this is NOT an april fools joke. Unless it turns out that I'm doing something extremely dumb. In which case, it is.

@cyrusd
Copy link

cyrusd commented Apr 1, 2010

double vs. float... floorf() is your friend :)

floorf(a*b) is 1699.000000

@darynnakhuda
Copy link

printf('%.16f',17.99 * (10**2))

1798.9999999999997726

That would be your problem :)

@sentientmonkey
Copy link
Author

So, the real reason I am mucking about with floor() is because that's how ruby's Fixnum.to_i function works...
http://www.ruby-doc.org/core/classes/Float.src/M000240.html

floorf sort of fixes it... for a certain threshold. Here's the output for with more precision (and some more debugging)

a is 16.989999999999998436805981327779591083526611328125000000000000000000000000000000000000000000000000000
b is 100.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
a_b is 1698.999999999999772626324556767940521240234375000000000000000000000000000000000000000000000000000000000
a_b < c
floorf(a*b) is 1699.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment