Skip to content

Instantly share code, notes, and snippets.

@fehiepsi

fehiepsi/pyro_trace_jit_bug.ipynb

Last active October 26, 2018 07:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save fehiepsi/e2cc69bfaa9b00033834756b3092970f to your computer and use it in GitHub Desktop.
Save fehiepsi/e2cc69bfaa9b00033834756b3092970f to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
pyro_trace_jit_bug.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@neerajprad
Copy link

With z being 0, is it possible that the probability of pyro.sample("obs", dist.Normal(3, z), obs=data) being close to 0 accounts for instability between the jit and nojit versions? The difference is large but both are essentially 0 on the probability scale. Can we see this difference for other inputs with a higher value for log_prob?

@fehiepsi
Copy link
Author

z = 1 indeed (after transform). You can remove the transform code and set z = 1, but precision is not the point I want to make here.

On probability scale, e^-1 is definitely different to e^-1000. If the number of data is 5000, then grad will be 5000x off!

@neerajprad
Copy link

z = 1 indeed (after transform)

Sorry, missed the transform code, and this indeed is surprising. Does the jit throw any warnings at all?

@fehiepsi
Copy link
Author

There is no warning at all.

@neerajprad
Copy link

Great to know, @fehiepsi! 👍 on the cool detective work. Moving the discussion back to the issue for better visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment