Skip to content

Instantly share code, notes, and snippets.

@ericjang
Created March 19, 2016 17:46
Show Gist options
  • Save ericjang/4e933e7321dba889d131 to your computer and use it in GitHub Desktop.
Save ericjang/4e933e7321dba889d131 to your computer and use it in GitHub Desktop.
KL Gaussian
-- torch implementation of KL between Gaussians
self.output = -1/2(
+ torch.cmul(iqv, p.sigma):sum() -- trace term, tr(\Sigma_q^{-1} * \Sigma_p)
+ torch.cmul(diff:clone():pow(2), iqv):sum() -- difference in means, (\mu_q-\mu_p)^T\Sigma_q^{-1}(\mu_q-\mu_p)
- ndim -- k
+ torch.log(q.sigma):sum() - torch.log(p.sigma):sum() -- ratio of determinants log |\Sigma_q| / |\Sigma_p|
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment