Skip to content

Instantly share code, notes, and snippets.

@digantamisra98
Last active August 12, 2019 00:25
Show Gist options
  • Save digantamisra98/aa8f003d6d698d6ec16476b8152228b7 to your computer and use it in GitHub Desktop.
Save digantamisra98/aa8f003d6d698d6ec16476b8152228b7 to your computer and use it in GitHub Desktop.
'''
Script provides functional interface for Mish activation function.
'''
# import pytorch
import torch
import torch.nn.functional as F
def mish(input):
'''
Applies the mish function element-wise:
mish(x) = x * tanh(softplus(x)) = x * tanh(ln(1 + exp(x)))
See additional documentation for mish class.
'''
return input * torch.tanh(F.softplus(input))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment