Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@springcoil

This comment has been minimized.

Copy link

commented Oct 11, 2016

Based on my limited understanding I don't see much wrong with the design. I'll dig into that Normalizing flows paper. I think this needs some fleshing out into a tutorial - right now it's not clear what the strengths and weaknesses of these methods are. I'm happy to write up some notes and send a PR though.

@taku-y

This comment has been minimized.

Copy link
Owner Author

commented Oct 11, 2016

Thanks for your comment! I also think this notebook needs better explanation. I will work on it.

@twiecki

This comment has been minimized.

Copy link

commented Oct 13, 2016

So the last cell is what I would need to create to run my model with NF-ADVI? How do I know what to pick for:

n_unitss = [4]
hs = (tt.nnet.sigmoid, tt.nnet.sigmoid)
gs = (tt.nnet.softplus, tt.exp)
iaf1 = SimpleMadeIAF(n_unitss, hs, gs, rev_order=False, random_seed=0)
iaf2 = SimpleMadeIAF(n_unitss, hs, gs, rev_order=True, random_seed=1)
nfs = [iaf2, iaf1]

?

Will that change for every model? Do I need a iaf for every RV?

@taku-y

This comment has been minimized.

Copy link
Owner Author

commented Oct 27, 2016

Sorry for late reply. nfs=[iaf2, iaf1] means that the concatenated vector of RVs, denoted by z, is transformed as z_new = iaf1(iaf2(z)). I need to explain about inverse autoregressive flows and API in this cell.

@springcoil

This comment has been minimized.

Copy link

commented Oct 27, 2016

Hi @taku-y thanks for adding Autoencoding Variational Bayes. I think it's fair to add this, I'll try to add in MADE soon, maybe with some helper writing to explain where all of this comes from. Right now it feels a bit disjointed to the rest of the writing.

@taku-y

This comment has been minimized.

Copy link
Owner Author

commented Oct 27, 2016

In the notebook, MADE is used as a model that satisfy the condition IAF requires. So I think IAF should be explained first. I will be on it when finish work.

@springcoil

This comment has been minimized.

Copy link

commented Dec 16, 2016

I'd say that when your code is ready add this in to the notebooks, we can add other things like IAF afterwards. What do you think @twiecki

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.