Skip to content

Instantly share code, notes, and snippets.

Last active Sep 30, 2017
What would you like to do?
Key Code Blocks of Pytorch RNN Dropout Implementation
class LockedDropout(nn.Module):
# ...
def forward(self, x, dropout=0.5):
if not or not dropout:
return x
m =, x.size(1), x.size(2)).bernoulli_(1 - dropout)
mask = Variable(m, requires_grad=False) / (1 - dropout)
mask = mask.expand_as(x)
return mask * x
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment