Last active
October 24, 2020 17:09
-
-
Save finlytics-hub/6b1b7cdef7dd6475e02d45a0fa88562c to your computer and use it in GitHub Desktop.
Linear Regression - Batch GD
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
''' | |
NORMAL EQUATION | |
''' | |
# generate some random numbers (independent variable) | |
X = 5 * np.random.rand(500,1) | |
# calculate linearly related (plus some noise) target variable in the form y = 10 + 2x + noise | |
y = 10 + 2 * X + np.random.randn(500,1) | |
# add ones to X for each observation (X0) | |
X_2d = np.c_[np.ones((500, 1)), X] | |
# calculate theta that minimizes MSE through Normal Equation | |
theta_best = np.linalg.inv(X_2d.T.dot(X_2d)).dot(X_2d.T).dot(y) | |
print('Normal Equation:\n', theta_best) | |
''' | |
BATCH GRADIENT DESCENT | |
''' | |
# set the learning rate | |
eta = 0.1 | |
# max number of iterations for GD to try and converge | |
n_iterations = 1000 | |
# number of observations in the training data, i.e. X | |
m = 500 | |
# random initialization | |
theta = np.random.randn(2, 1) | |
# perform iterations | |
for iteration in range(n_iterations): | |
gradients = 2/m * X_2d.T.dot(X_2d.dot(theta) - y) | |
theta = theta - eta * gradients | |
# print theta values | |
print('Batch GD:\n', theta) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment