So the last cell is what I would need to create to run my model with NF-ADVI? How do I know what to pick for:
n_unitss = [4]
hs = (tt.nnet.sigmoid, tt.nnet.sigmoid)
gs = (tt.nnet.softplus, tt.exp)
iaf1 = SimpleMadeIAF(n_unitss, hs, gs, rev_order=False, random_seed=0)
iaf2 = SimpleMadeIAF(n_unitss, hs, gs, rev_order=True, random_seed=1)
nfs = [iaf2, iaf1]
?
Will that change for every model? Do I need a iaf for every RV?
Sorry for late reply. nfs=[iaf2, iaf1] means that the concatenated vector of RVs, denoted by z, is transformed as z_new = iaf1(iaf2(z)). I need to explain about inverse autoregressive flows and API in this cell.
Hi @taku-y thanks for adding Autoencoding Variational Bayes. I think it's fair to add this, I'll try to add in MADE soon, maybe with some helper writing to explain where all of this comes from. Right now it feels a bit disjointed to the rest of the writing.
In the notebook, MADE is used as a model that satisfy the condition IAF requires. So I think IAF should be explained first. I will be on it when finish work.
I'd say that when your code is ready add this in to the notebooks, we can add other things like IAF afterwards. What do you think @twiecki
Thanks for your comment! I also think this notebook needs better explanation. I will work on it.