@taku-y Thanks for sharing. I have not tried your code but reading it I think there is bug here:
don't you need to enforce non-negativity for
@taku-y Thanks for sharing this. I've been working on this problem for a while myself, so it is a relief to see it work.
had to change
most importantly, had to change the learning rate. i tried values in the range
As such my runs never reach the
Topic #1: people like just does new use good time know think
Topic #2: like just people know does don good new use think
Topic #3: people know just like don does good time use god
Topic #4: people know time like use don does just good god
Topic #5: just like don time think new say know people good
Topic #6: don just time know good think use people like make
Topic #7: just people know edu use like good don does think
Topic #8: people new does time just don good use like know
Topic #9: like don just new good people know does time use`
is there anything that pops out to you about my changes that might be causing this behaviour? in particular, it seems like most of the topics are quite similar in my case.
@akashgit Sorry for late to reply. I had been working on incorporation of autoencoding VB into the PyMC3 repo.
For 1. and 2. of your comments, I fixed the bug on the notebook. Your comments were helpful for the fix. Thanks.
hi @taku-y, I tried using this notebook after updating to the latest pymc3 version (instead of running it from your fork) and I keep getting
Hi, thanks for your report. What version of theano and numpy are you using? I use theano 0.8.2 and numpy 1.10.4. Updating these libraries to latest versions might resolve the problem. Actually, I'm using a Docker container. If you want, I prepare a Docker file to build a container image on which the notebook works.
Hi @taku-y , I was using theano 0.8.2 and 1.10.4 before updating to latest pymc3 (when things were working). After upgrading, while debugging through the error i ended up on 0.9.0.dev2 and numpy 1.11.1 (though it wasn't helpful).
The interesting thing is that these problem only occur when i update to the latest pymc3 version. Running them on your old fork (from the time of my first comment with the suggested edits) doesn't cause any of those errors.
i haven't used docker before, but m willing to try it so it would be nice if you could send me the image. Thanks again.
Hi @taku-y, Here is the actual error. Somehow magically float64 are appearing even though my config file and flags are appropriately set.
Applied stickbreaking-transform to theta and added transformed theta_stickbreaking_ to model.
Applied stickbreaking-transform to beta and added transformed beta_stickbreaking_ to model.
TypeError Traceback (most recent call last)
/IPC_MAP/lib/python2.7/site-packages/pymc3-3.0-py2.7.egg/pymc3/distributions/distribution.pyc in new(cls, name, _args, *_kwargs)
/IPC_MAP/lib/python2.7/site-packages/pymc3-3.0-py2.7.egg/pymc3/model.pyc in Var(self, name, dist, data)
/IPC_MAP/lib/python2.7/site-packages/pymc3-3.0-py2.7.egg/pymc3/model.pyc in init(self, type, owner, index, name, data, distribution, model)
/IPC_MAP/lib/python2.7/site-packages/theano/tensor/var.pyc in getitem(self, args)
/IPC_MAP/lib/python2.7/site-packages/theano/tensor/var.pyc in take(self, indices, axis, mode)
/IPC_MAP/lib/python2.7/site-packages/theano/tensor/subtensor.pyc in take(a, indices, axis, mode)
/IPC_MAP/lib/python2.7/site-packages/theano/gof/op.pyc in call(self, _inputs, *_kwargs)
/IPC_MAP/lib/python2.7/site-packages/theano/gof/op.pyc in _get_test_value(cls, v)
/IPC_MAP/lib/python2.7/site-packages/theano/tensor/type.pyc in filter(self, data, strict, allow_downcast)
The error when converting the test value to that variable type:
TensorType(float32, matrix) cannot store a value of dtype float64 without risking loss of precision. If you do not mind this loss, you can: 1) explicitly cast your data to float32, or 2) set "allow_input_downcast=True" when calling "function".
Hi @akashgit, thanks for sending me detailed information. But I could not reproduce your result yet. I tested the notebook on theano-0.8.2 and 0.9.0dev2, python2 and 3. Under all of these environments, ADVI worked.
Here is my Dockerfile on which the notebook ran.
From jupyter/datascience-notebook MAINTAINER Taku Yoshioka <email@example.com> ENV TERM xterm-color USER jovyan RUN pip install theano joblib
Save the above as
You can access the notebook server via
Here is the option of the command