Skip to content

Instantly share code, notes, and snippets.

@snickerbockers
Created July 6, 2020 01:49
Show Gist options
  • Save snickerbockers/919c88e3607333840430f552f8c3ce38 to your computer and use it in GitHub Desktop.
Save snickerbockers/919c88e3607333840430f552f8c3ce38 to your computer and use it in GitHub Desktop.
simple script for converting a floating point to a fixed point with 16 bits of precision
#!/usr/bin/env python
import sys
N_FRAC_BITS = 16
val = val_orig = float(sys.argv[1])
frac_part = 0
for i in range(N_FRAC_BITS):
weight = 1.0 / (1 << i)
if (weight <= val):
val -= weight
frac_part |= (1 << (N_FRAC_BITS - i))
print "val was %f" % val_orig
print "frac_part is 0x%x" % frac_part
floating = frac_part / float(1 << N_FRAC_BITS)
print "converted back, that's %f" % floating
print "the error is %f" % abs(val_orig - floating)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment