Skip to content

Instantly share code, notes, and snippets.

@y0ast
Last active May 30, 2021 21:01
Show Gist options
  • Save y0ast/f69966e308e549f013a92dc66debeeb4 to your computer and use it in GitHub Desktop.
Save y0ast/f69966e308e549f013a92dc66debeeb4 to your computer and use it in GitHub Desktop.
@y0ast
Copy link
Author

y0ast commented May 22, 2021

Unfortunately that does not give the correct behavior: you're not randomizing your batches at each epoch which leads to significant reduced performance.

Yes this normalization is 0 mean, 1 std, for a VAE + MNIST you generally model your data as a multivariate bernoulli, which requires it to be between 0 and 1.

@AlexPasqua
Copy link

Unfortunately that does not give the correct behavior: you're not randomizing your batches at each epoch which leads to significant reduced performance.

That's true, the shuffling should then be done manually. I think this should work:

train_dataset.data = train_dataset.data[torch.randperm(train_dataset.data.shape[0])]

(assuming the first dimension of train_dataset.data to be the batch size)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment