dict_learning_errorcalculation.py

import numpy as np
from sklearn.decomposition import DictionaryLearning

nfeat = 20

X = np.random.random(nfeat)
X = X.reshape(1,-1)#   otherwise sklearn complains

alpha = 1
n_comp = 4

dictlearn = DictionaryLearning(n_components=n_comp,alpha=alpha,verbose=2)

sk_code = dictlearn.fit_transform(X)
sk_dictionary = dictlearn.components_
sk_error = dictlearn.error_[-1]

my_estim = np.dot(sk_code,sk_dictionary)-X

my_residuals = np.linalg.norm(my_estim,2)**2

my_error = 0.5*my_residuals + (alpha)*np.sum(np.abs(sk_code))

print "Error : my estimation : %f" % my_error
print "Error : sklearn estimation : %f" % sk_error

print "Difference : %f" % np.abs(my_error - sk_error)

print "1 / (2*(alpha**2)) = %f" % (1./2*(alpha**2))

The formula to calculate the error is the one specified here

http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.DictionaryLearning.html

Note that in sklearn's code the same formula is also implemented :

https://github.com/scikit-learn/scikit-learn/blob/51a765a/sklearn/decomposition/dict_learning.py#L527

Not with more samples to train on (e.g., nsamples = 30):

Error : my estimation : 90.230173
Error : sklearn estimation : 72.214592
Difference : 18.015580
1 / (2*(alpha**2)) = 0.500000

Ok, thanks, hmm..
Can you provide me your X so that I can run the same here ?

Now trying over by directly using the low level function - it seems to work better as I directly control the outputs.

`import numpy as np
from sklearn.decomposition import DictionaryLearning

from sklearn.decomposition import dict_learning

nfeat = 10

X = np.random.random(nfeat)
X = X.reshape(nfeat/10,10)# otherwise sklearn complains

alpha = 0.01
n_comp = 8

sk_code,sk_dictionary,sk_allerrors = dict_learning(X,n_comp,alpha,verbose=1)

sk_error = sk_allerrors[-1]

my_estim = np.dot(sk_code,sk_dictionary)-X

my_residuals = np.linalg.norm(my_estim,2)**2

my_error = 0.5_my_residuals + (alpha)_np.sum(np.abs(sk_code))

print "Error : my estimation : %f" % my_error
print "Error : sklearn estimation : %f" % sk_error

print "Difference : %f" % np.abs(my_error - sk_error)`