Skip to content

Instantly share code, notes, and snippets.

@kbrafford
Last active April 5, 2020 10:56
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kbrafford/e0115e796890fcefb4f0c35248bd05f1 to your computer and use it in GitHub Desktop.
Save kbrafford/e0115e796890fcefb4f0c35248bd05f1 to your computer and use it in GitHub Desktop.
>>> for p in (3,6,9):
... EFloat.precision = p
... for f in (math.pi, 12, 823, 1024, 1024**2, 1024**3):
... print EFloat(f), 1/EFloat(f)
...
3.142e+00 318.310e-03
12.000e+00 83.333e-03
823.000e+00 1.215e-03
1.024e+03 976.562e-06
1.049e+06 953.674e-09
1.074e+09 931.323e-12
3.141593e+00 318.309886e-03
12.000000e+00 83.333333e-03
823.000000e+00 1.215067e-03
1.024000e+03 976.562500e-06
1.048576e+06 953.674316e-09
1.073742e+09 931.322575e-12
3.141592654e+00 318.309886184e-03
12.000000000e+00 83.333333333e-03
823.000000000e+00 1.215066829e-03
1.024000000e+03 976.562500000e-06
1.048576000e+06 953.674316406e-09
1.073741824e+09 931.322574615e-12
@stroobandt
Copy link

stroobandt commented Dec 6, 2016

Dear Keith, that is exactly how it should be! (I cross-checked with a HP calculator to make sure.)
Hopefully, somebody will pick this Python issue up now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment