Skip to content

Instantly share code, notes, and snippets.

@egonSchiele
Created February 22, 2016 03:29
Show Gist options
  • Save egonSchiele/5aba0e1b9f77bd3f6afa to your computer and use it in GitHub Desktop.
Save egonSchiele/5aba0e1b9f77bd3f6afa to your computer and use it in GitHub Desktop.
% my training data.
% so if x > 3 || x < 7, y = 1, otherwise y = 0.
x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
y = [0, 0, 0, 1, 1, 1, 0, 0, 0, 0];
% instead of theta' * x, I'm trying to create
% a non-linear decision boundary.
function result = h(x, theta)
result = sigmoid(theta(1) + theta(2) * x + theta(3) * ((x - theta(4))^2));
end
function result = sigmoid(z)
result = 1 / (1 + e ^ (-z));
end
% cost function, works correctly
function distance = cost(theta, x, y)
distance = 0;
for i = 1:length(x) % arrays in octave are indexed starting at 1
if (y(i) == 1)
distance += -log(h(x(i), theta));
else
distance += -log(1 - h(x(i), theta));
end
end
% get how far off we were on average
distance = distance / length(x);
end
alpha = 0.1;
iters = 3000;
m = length(x);
% initial values
theta = [0, 0, 0, 0];
% this should calculate values close to
% theta = [1, 3, 10, 5],
% but it does not
% run gradient descent
for i = 1:iters
h_all = [];
for j = 1:length(x)
h_all = [h_all, h(x(j), theta)];
end
% pad the x vector since we have four thetas
padded_x = [ones(1, length(x)); x ; x ; x];
theta -= alpha .* 1/m .* ((h_all - y) * padded_x');
% cost does NOT keep going down here so there's definitely a bug somewhere.
disp([theta, cost(theta, x, y)]);
end
% for each number between 1 and 10,
% print out the probability that y(i) = 1
for i = 1:10
disp([i, h(i, theta) * 100]);
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment