Skip to content

Instantly share code, notes, and snippets.

@taku-y
Last active August 8, 2021 06:16
Show Gist options
  • Save taku-y/43550e688f4020ac7da15a181d262f2f to your computer and use it in GitHub Desktop.
Save taku-y/43550e688f4020ac7da15a181d262f2f to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@springcoil
Copy link

Based on my limited understanding I don't see much wrong with the design. I'll dig into that Normalizing flows paper. I think this needs some fleshing out into a tutorial - right now it's not clear what the strengths and weaknesses of these methods are. I'm happy to write up some notes and send a PR though.

@taku-y
Copy link
Author

taku-y commented Oct 11, 2016

Thanks for your comment! I also think this notebook needs better explanation. I will work on it.

@twiecki
Copy link

twiecki commented Oct 13, 2016

So the last cell is what I would need to create to run my model with NF-ADVI? How do I know what to pick for:

n_unitss = [4]
hs = (tt.nnet.sigmoid, tt.nnet.sigmoid)
gs = (tt.nnet.softplus, tt.exp)
iaf1 = SimpleMadeIAF(n_unitss, hs, gs, rev_order=False, random_seed=0)
iaf2 = SimpleMadeIAF(n_unitss, hs, gs, rev_order=True, random_seed=1)
nfs = [iaf2, iaf1]

?

Will that change for every model? Do I need a iaf for every RV?

@taku-y
Copy link
Author

taku-y commented Oct 27, 2016

Sorry for late reply. nfs=[iaf2, iaf1] means that the concatenated vector of RVs, denoted by z, is transformed as z_new = iaf1(iaf2(z)). I need to explain about inverse autoregressive flows and API in this cell.

@springcoil
Copy link

Hi @taku-y thanks for adding Autoencoding Variational Bayes. I think it's fair to add this, I'll try to add in MADE soon, maybe with some helper writing to explain where all of this comes from. Right now it feels a bit disjointed to the rest of the writing.

@taku-y
Copy link
Author

taku-y commented Oct 27, 2016

In the notebook, MADE is used as a model that satisfy the condition IAF requires. So I think IAF should be explained first. I will be on it when finish work.

@springcoil
Copy link

I'd say that when your code is ready add this in to the notebooks, we can add other things like IAF afterwards. What do you think @twiecki

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment