Skip to content

Instantly share code, notes, and snippets.

@kenwebb
Last active January 8, 2018 21:46
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save kenwebb/dc8e65ab77b4855bc3d927aa2e511777 to your computer and use it in GitHub Desktop.
BRIIM - JavaScript in Machine Learning
<?xml version="1.0" encoding="UTF-8"?>
<!--Xholon Workbook http://www.primordion.com/Xholon/gwt/ MIT License, Copyright (C) Ken Webb, Mon Jan 08 2018 16:46:17 GMT-0500 (EST)-->
<XholonWorkbook>
<Notes><![CDATA[
Xholon
------
Title: BRIIM - JavaScript in Machine Learning
Description:
Url: http://www.primordion.com/Xholon/gwt/
InternalName: dc8e65ab77b4855bc3d927aa2e511777
Keywords:
My Notes
--------
January 6, 2018
use XholonMathjs.html
http://127.0.0.1:8888/XholonMathjs.html?app=BRIIM+-+JavaScript+in+Machine+Learning&src=lstr&gui=clsc
http://127.0.0.1:8888/XholonMathjs.html?app=dc8e65ab77b4855bc3d927aa2e511777&src=gist&gui=clsc
NvrtLnrRgrsnGrdntDscntbehavior .act() does not work
- TODO
- go to Chrome Developer Tools
- right-click the output from console.log(mtrx);
- select "Store as global variable"
- creates new var temp1
- run code:
var matrix = temp1;
let X = math.eval('matrix[:, 1]', {
matrix,
});
let y = math.eval('matrix[:, 2]', {
matrix,
});
COMLETE CODE
------------
var matrix = temp1;
function init(matrix) {
// Part 0: Preparation
console.log('Part 0: Preparation ...\n');
let X = math.eval('matrix[:, 1]', {
matrix,
});
let y = math.eval('matrix[:, 2]', {
matrix,
});
let m = y.length;
// Part 1: Cost
console.log('Part 1: Cost ...\n');
// Add Intercept Term
X = math.concat(math.ones([m, 1]).valueOf(), X);
let theta = [[-1], [2]];
let J = computeCost(X, y, theta);
console.log('Cost: ', J);
console.log('with: ', theta);
console.log('\n');
theta = [[0], [0]];
J = computeCost(X, y, theta);
console.log('Cost: ', J);
console.log('with: ', theta);
console.log('\n');
// Part 2: Gradient Descent
console.log('Part 2: Gradient Descent ...\n');
const ITERATIONS = 1500;
const ALPHA = 0.01;
theta = gradientDescent(X, y, theta, ALPHA, ITERATIONS);
console.log('theta: ', theta);
}
function computeCost(X, y, theta) {
let m = y.length;
let predictions = math.eval('X * theta', {
X,
theta,
});
let sqrErrors = math.eval('(predictions - y).^2', {
predictions,
y,
});
let J = math.eval(`1 / (2 * m) * sum(sqrErrors)`, {
m,
sqrErrors,
});
return J;
}
function gradientDescent(X, y, theta, ALPHA, ITERATIONS) {
let m = y.length;
let thetaZero = theta[0];
let thetaOne = theta[1];
for (let i = 0; i < ITERATIONS; i++) {
let predictions = math.eval('X * theta', {
X,
theta: [thetaZero, thetaOne],
});
thetaZero = math.eval(`thetaZero - ALPHA * (1 / m) * sum(predictions - y)`, {
thetaZero,
ALPHA,
m,
predictions,
y,
});
thetaOne = math.eval(`thetaOne - ALPHA * (1 / m) * sum((predictions - y)' * X[:, 2])`, {
thetaOne,
ALPHA,
m,
predictions,
y,
X,
});
}
return [thetaZero, thetaOne];
}
function gradientDescentAlternative(X, y, theta, ALPHA, ITERATIONS) {
let m = y.length;
let thetaZero = theta[0];
let thetaOne = theta[1];
for (let i = 0; i < ITERATIONS; i++) {
let predictions = math.eval('X * theta', {
X,
theta: [thetaZero, thetaOne],
});
thetaZero = math.eval(`thetaZero - ALPHA * (1 / m) * sum((predictions - y) .* X[:, 1])`, {
thetaZero,
ALPHA,
m,
predictions,
y,
X,
});
thetaOne = math.eval(`thetaOne - ALPHA * (1 / m) * sum((predictions - y) .* X[:, 2])`, {
thetaOne,
ALPHA,
m,
predictions,
y,
X,
});
}
return [thetaZero, thetaOne];
}
--------------------------------- end of COMPLETE CODE
- this works
test 1
------
var A = [1,2,3,4,5];
var b = 3;
math.eval('A + b', {A,b}); // [4, 5, 6, 7, 8]
math.eval('A * b', {A,b}); // [3, 6, 9, 12, 15]
References
----------
(1) https://github.com/javascript-machine-learning/organization-overview
BRIIM - JavaScript in Machine Learning
(2) https://github.com/javascript-machine-learning/linear-regression-gradient-descent
Linear Regression with Gradient Descent in JavaScript (Unvectorized, Visualized)
(3) https://www.robinwieruch.de/linear-regression-gradient-descent-javascript/
Linear Regression with Gradient Descent in JavaScript
Recently I started to take the Machine Learning course by Andrew Ng on Coursera.
In the following article, I want to guide you through building a linear regression with gradient descent algorithm in JavaScript.
Since JavaScript is the programming language that I feel most comfortable with, I try to apply my learnings in machine learning in JavaScript as long as I can.
Afterward, I hope to find the time to transition these learnings to Python.
(4) https://github.com/javascript-machine-learning/linear-regression-gradient-descent/blob/master/src/App.js
uses React; 134 lines of code
I will NOT use React in my version
(5) http://mathjs.org/
An extensive math library for JavaScript and Node.js
Math.js is an extensive math library for JavaScript and Node.js.
It features a flexible expression parser with support for symbolic computation,
comes with a large set of built-in functions and constants,
and offers an integrated solution to work with different data types like numbers, big numbers, complex numbers, fractions, units, and matrices.
Powerful and easy to use.
(6) https://github.com/javascript-machine-learning/univariate-linear-regression-gradient-descent-javascript
Univariate Linear Regression with Gradient Descent in JavaScript (Vectorized)
(7) https://github.com/javascript-machine-learning/univariate-linear-regression-gradient-descent-javascript/blob/master/src/data.csv
97 rows of data
]]></Notes>
<_-.XholonClass>
<PhysicalSystem/>
<LnrRgrsnGrdntDscnt/> <!-- Linear Regression with Gradient Descent in JavaScript -->
<NvrtLnrRgrsnGrdntDscnt/> <!-- Univariate Linear Regression with Gradient Descent in JavaScript (Vectorized) -->
</_-.XholonClass>
<xholonClassDetails>
</xholonClassDetails>
<PhysicalSystem>
<LnrRgrsnGrdntDscnt/>
<!-- data is from source [7] -->
<NvrtLnrRgrsnGrdntDscnt><Attribute_String>
6.1101,17.592
5.5277,9.1302
8.5186,13.662
7.0032,11.854
5.8598,6.8233
8.3829,11.886
7.4764,4.3483
8.5781,12
6.4862,6.5987
5.0546,3.8166
5.7107,3.2522
14.164,15.505
5.734,3.1551
8.4084,7.2258
5.6407,0.71618
5.3794,3.5129
6.3654,5.3048
5.1301,0.56077
6.4296,3.6518
7.0708,5.3893
6.1891,3.1386
20.27,21.767
5.4901,4.263
6.3261,5.1875
5.5649,3.0825
18.945,22.638
12.828,13.501
10.957,7.0467
13.176,14.692
22.203,24.147
5.2524,-1.22
6.5894,5.9966
9.2482,12.134
5.8918,1.8495
8.2111,6.5426
7.9334,4.5623
8.0959,4.1164
5.6063,3.3928
12.836,10.117
6.3534,5.4974
5.4069,0.55657
6.8825,3.9115
11.708,5.3854
5.7737,2.4406
7.8247,6.7318
7.0931,1.0463
5.0702,5.1337
5.8014,1.844
11.7,8.0043
5.5416,1.0179
7.5402,6.7504
5.3077,1.8396
7.4239,4.2885
7.6031,4.9981
6.3328,1.4233
6.3589,-1.4211
6.2742,2.4756
5.6397,4.6042
9.3102,3.9624
9.4536,5.4141
8.8254,5.1694
5.1793,-0.74279
21.279,17.929
14.908,12.054
18.959,17.054
7.2182,4.8852
8.2951,5.7442
10.236,7.7754
5.4994,1.0173
20.341,20.992
10.136,6.6799
7.3345,4.0259
6.0062,1.2784
7.2259,3.3411
5.0269,-2.6807
6.5479,0.29678
7.5386,3.8845
5.0365,5.7014
10.274,6.7526
5.1077,2.0576
5.7292,0.47953
5.1884,0.20421
6.3557,0.67861
9.7687,7.5435
6.5159,5.3436
8.5172,4.2415
9.1802,6.7981
6.002,0.92695
5.5204,0.152
5.0594,2.8214
5.7077,1.8451
7.6366,4.2959
5.8707,7.2029
5.3054,1.9869
8.2934,0.14454
13.394,9.0551
5.4369,0.61705
</Attribute_String></NvrtLnrRgrsnGrdntDscnt>
</PhysicalSystem>
<LnrRgrsnGrdntDscntbehavior implName="org.primordion.xholon.base.Behavior_gwtjs"><![CDATA[
// adjust training set size
const M = 10;
// generate random training set
const DATA = [];
const getRandomIntFromInterval = (min, max) =>
Math.floor(Math.random() * (max - min + 1) + min);
const createRandomPortlandHouse = () => ({
squareMeter: getRandomIntFromInterval(0, 100),
price: getRandomIntFromInterval(0, 100),
});
for (let i = 0; i < M; i++) {
DATA.push(createRandomPortlandHouse());
}
const x = DATA.map(date => date.squareMeter);
const y = DATA.map(date => date.price);
// linear regression and gradient descent
const LEARNING_RATE = 0.0003;
let thetaOne = 0;
let thetaZero = 0;
const hypothesis = (x) => {
return thetaZero + thetaOne * x;
}
const learn = (alpha) => {
let thetaZeroSum = 0;
let thetaOneSum = 0;
for (let i = 0; i < M; i++) {
thetaZeroSum += hypothesis(x[i]) - y[i];
thetaOneSum += (hypothesis(x[i]) - y[i]) * x[i];
}
thetaZero = thetaZero - (alpha / M) * thetaZeroSum;
thetaOne = thetaOne - (alpha / M) * thetaOneSum;
}
const cost = () => {
let sum = 0;
for (let i = 0; i < M; i++) {
sum += Math.pow(hypothesis(x[i]) - y[i], 2);
}
return sum / (2 * M);
}
// count iterations
let iteration = 0;
////////////////////////////////////////// the standard Xholon code
var me, beh = {
postConfigure: function() {
me = this.cnode.parent();
},
act: function() {
$wnd.console.log(M);
learn(LEARNING_RATE);
iteration++;
me.println(cost().toFixed(2));
}
}
//# sourceURL=doLnrRgrsnGrdntDscnt.js
]]></LnrRgrsnGrdntDscntbehavior>
<NvrtLnrRgrsnGrdntDscntbehavior implName="org.primordion.xholon.base.Behavior_gwtjs"><![CDATA[
/*
function init(matrix) {
// Part 0: Preparation
console.log('Part 0: Preparation ...\n');
let X = math.eval('matrix[:, 1]', {
matrix,
});
let y = math.eval('matrix[:, 2]', {
matrix,
});
}
*/
////////////////////////////////////////// the standard Xholon code
var me, mtrx, math, beh = {
postConfigure: function() {
me = this.cnode.parent();
//$wnd.xh.require("math"); // the library is already part of XholonMathjs.html
var attrNode = me.xpath("Attribute_String")
var matrixStr = attrNode.text();
me.println(matrixStr);
mtrx = [];
var arr = matrixStr.split("\n");
for (var i = 0; i < arr.length; i++) {
var row = arr[i];
var rowItemsArr = row.split(",");
var rowArr = [];
rowArr.push(rowItemsArr[0]);
rowArr.push(rowItemsArr[1]);
mtrx.push(rowArr);
}
$wnd.console.log(mtrx);
attrNode.remove();
},
act: function() {
/*
math = $wnd.math;
me.println(math.sqrt(-4)); // 2i
//$wnd.xh.matrix = mtrx;
//init($wnd.xh.matrix);
var matrixObj = {};
matrixObj.matrix = mtrx;
//init(matrixObj);
*/
}
}
//# sourceURL=doNvrtLnrRgrsnGrdntDscnt.js
]]></NvrtLnrRgrsnGrdntDscntbehavior>
<SvgClient><Attribute_String roleName="svgUri"><![CDATA[data:image/svg+xml,
<svg width="100" height="50" xmlns="http://www.w3.org/2000/svg">
<g>
<title>Linear Regression with Gradient Descent in JavaScript</title>
<rect id="PhysicalSystem/LnrRgrsnGrdntDscnt" fill="#98FB98" height="50" width="50" x="25" y="0"/>
</g>
</svg>
]]></Attribute_String><Attribute_String roleName="setup">${MODELNAME_DEFAULT},${SVGURI_DEFAULT}</Attribute_String></SvgClient>
</XholonWorkbook>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment