Skip to content

Instantly share code, notes, and snippets.

@MaxWilson
Created March 30, 2023 00:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save MaxWilson/fa57042f7a21cb5844ccc61acc4b9860 to your computer and use it in GitHub Desktop.
Save MaxWilson/fa57042f7a21cb5844ccc61acc4b9860 to your computer and use it in GitHub Desktop.
F# code for computing rounding errors in STRoll
let fstroll x = (System.Math.Log2((x)/10.)*10.+10.)
let stroll x = fstroll (float x)
let roundedStroll x = (stroll x |> System.Math.Round |> int)
let round (x:float) = x |> System.Math.Round |> int
let normalize y x =
let bigger = max y x
let smaller = min y x
(10. * (float bigger / float smaller))
for x in [1..25] do
for y in [x+1..25] do
let roundedMargin = (roundedStroll y - roundedStroll x)
let rawMargin = (fstroll (normalize y x)) - 10.
if round rawMargin <> roundedMargin then
printfn $"ST {x} vs. {y}: Rounded margin %+d{ roundedMargin }. Should be %+d{ round rawMargin } (%+.2f{rawMargin})"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment