Skip to content

Instantly share code, notes, and snippets.

Last active December 6, 2023 06:25
Show Gist options
  • Save danoneata/9927923 to your computer and use it in GitHub Desktop.
Save danoneata/9927923 to your computer and use it in GitHub Desktop.
Fisher vectors with sklearn
import numpy as np
import pdb
from sklearn.datasets import make_classification
from sklearn.mixture import GaussianMixture as GMM
def fisher_vector(xx, gmm):
"""Computes the Fisher vector on a set of descriptors.
xx: array_like, shape (N, D) or (D, )
The set of descriptors
gmm: instance of sklearn mixture.GMM object
Gauassian mixture model of the descriptors.
fv: array_like, shape (K + 2 * D * K, )
Fisher vector (derivatives with respect to the mixing weights, means
and variances) of the given descriptors.
J. Krapac, J. Verbeek, F. Jurie. Modeling Spatial Layout with Fisher
Vectors for Image Categorization. In ICCV, 2011.
xx = np.atleast_2d(xx)
N = xx.shape[0]
# Compute posterior probabilities.
Q = gmm.predict_proba(xx) # NxK
# Compute the sufficient statistics of descriptors.
Q_sum = np.sum(Q, 0)[:, np.newaxis] / N
Q_xx =, xx) / N
Q_xx_2 =, xx ** 2) / N
# Compute derivatives with respect to mixing weights, means and variances.
d_pi = Q_sum.squeeze() - gmm.weights_
d_mu = Q_xx - Q_sum * gmm.means_
d_sigma = (
- Q_xx_2
- Q_sum * gmm.means_ ** 2
+ Q_sum * gmm.covariances_
+ 2 * Q_xx * gmm.means_)
# Merge derivatives into a vector.
return np.hstack((d_pi, d_mu.flatten(), d_sigma.flatten()))
def main():
# Short demo.
K = 64
N = 1000
xx, _ = make_classification(n_samples=N)
xx_tr, xx_te = xx[: -100], xx[-100: ]
gmm = GMM(n_components=K, covariance_type='diag')
fv = fisher_vector(xx_te, gmm)
if __name__ == '__main__':
Copy link

Thank you so much for the comprehensive answer.
I really appreciate that.

Copy link

Hai I'm beginner so i don't know working of fisher vector encoding. Please help to understand this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment