Skip to content

Instantly share code, notes, and snippets.

@cwharland
Created January 17, 2016 23:23
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cwharland/66dcb10b7d7605a96413 to your computer and use it in GitHub Desktop.
Save cwharland/66dcb10b7d7605a96413 to your computer and use it in GitHub Desktop.
Example of skflow's inability to recover weights?
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
import skflow
import pytest
class TestLinearRegression:
def setup_class(self):
'''Simple linear model with a tiny amount of noise'''
rng = np.random.RandomState(67)
N = 1000
n_weights = 10
self.bias = 2
self.X = rng.uniform(-1, 1, (N, n_weights))
self.weights = 10 * rng.randn(n_weights)
self.y = np.dot(self.X, self.weights)
self.y += rng.randn(len(self.X)) * 0.05 + rng.normal(self.bias, 0.01)
def test_skflow(self):
'''Check that skflow LinearRegression can recover weights and bias'''
print('Fitting skflow model...')
regressor = skflow.TensorFlowLinearRegressor()
regressor.fit(self.X, self.y)
# Have to flatten weights since they come in (X, 1) shape
np.testing.assert_allclose(self.weights,
regressor.weights_.flatten(),
rtol=0.01)
assert abs(self.bias - regressor.bias_) < 0.1
def test_sklearn(self):
'''Check that sklearn LinearRegression can recover weights and bias'''
print('Fitting sklearn model...')
regressor = LinearRegression()
regressor.fit(self.X, self.y)
np.testing.assert_allclose(self.weights,
regressor.coef_.flatten(),
rtol=0.01)
assert abs(self.bias - regressor.intercept_) < 0.1
@cwharland
Copy link
Author

scikit-learn LinearRegression passes just fine but skflow fails with:

==================================== FAILURES ==============================
______________________________________TestLinearRegression.test_skflow _______________________________

self = <skflow_test.TestLinearRegression object at 0x10ad04400>

    def test_skflow(self):
        '''Check that skflow LinearRegression can recover weights and bias'''
        print('Fitting skflow model...')
        regressor = skflow.TensorFlowLinearRegressor()
        regressor.fit(self.X, self.y)
        # Have to flatten weights since they come in (X, 1) shape
        np.testing.assert_allclose(self.weights,
                                   regressor.weights_.flatten(),
>                                  rtol=0.01)
E       AssertionError:
E       Not equal to tolerance rtol=0.01, atol=0
E
E       (mismatch 100.0%)
E        x: array([-10.296441,   2.790587,   2.11216 ,  10.949834,   6.409751,
E               10.812394, -19.089351,  -1.307512,   7.00129 ,   5.26752 ])
E        y: array([-0.366528, -0.048527, -0.021612,  0.400875,  0.100367,  0.514881,
E              -0.439183, -0.038739,  0.150159,  0.116843], dtype=float32)

skflow_test.py:27: AssertionError
---------------------------------------- Captured stdout call -------------------------------------
Fitting skflow model...
Step #1, avg. loss: 367.39044
Step #6, avg. loss: 335.57275
Step #11, avg. loss: 252.00935
Step #16, avg. loss: 273.52588
Step #21, avg. loss: 272.40781
Step #26, avg. loss: 240.66650
Step #31, avg. loss: 254.82413
Step #36, avg. loss: 225.10262
Step #41, avg. loss: 257.79672
Step #46, avg. loss: 238.27005
--------------------------------------- Captured stderr call -----------------------------------------
I tensorflow/core/common_runtime/local_device.cc:40] Local device intra op parallelism threads: 4
I tensorflow/core/common_runtime/direct_session.cc:58] Direct session inter op parallelism threads: 4
================================ 1 failed, 1 passed in 1.55 seconds =============================

The weights are way off (first are true weights, second are skflow). Doesn't seem to be fixed with standard scaling so I must be missing something.

Package details:
Python 3.5
tensorflow 0.6.0
skflow 0.0.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment