Skip to content

Instantly share code, notes, and snippets.

@coppeliaMLA

coppeliaMLA/confusion.htm

Created June 24, 2014 07:52
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save coppeliaMLA/7447bdfc6cb8e6e206fe to your computer and use it in GitHub Desktop.
Save coppeliaMLA/7447bdfc6cb8e6e206fe to your computer and use it in GitHub Desktop.
Download ZIP
Exploration of a confusion matrix using tangle.js
Raw
confusion.htm
<!DOCTYPE html>
<html>
<head>
<title>Tangle: a JavaScript library for reactive documents</title>
<link rel="stylesheet" href="http://worrydream.com/Tangle/TangleKit/TangleKit.css" type="text/css">
<script type="text/javascript" src="http://worrydream.com/Tangle/TangleKit/mootools.js"></script>
<script type="text/javascript" src="http://worrydream.com/Tangle/TangleKit/sprintf.js"></script>
<script type="text/javascript" src="http://worrydream.com/Tangle/TangleKit/BVTouchable.js"></script>
<script type="text/javascript" src="http://worrydream.com/Tangle/Tangle.js"></script>
<script type="text/javascript" src="http://worrydream.com/Tangle/TangleKit/TangleKit.js"></script>
<script type="text/javascript"
src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<style>
body {
margin: 50px;
padding: 0px;
font-family: verdana, arial, helvetica, sans-serif;
color: #333;
background-color: white;
}
h1 {
margin: 50px 0px 15px 0px;
padding: 0px;
font-size: 28px;
line-height: 28px;
font-weight: 900;
color: #ccc;
}
p {
font: 12px/20px verdana, arial, helvetica, sans-serif;
margin: 0px 0px 16px 0px;
padding: 0px;
}
#Content > p {
margin: 0px;
}
#Content > p+ p {
text-indent: 30px;
}
a {
color: #09c;
font-size: 11px;
text-decoration: none;
font-weight: 600;
font-family: verdana, arial, helvetica, sans-serif;
}
a:link {
color: #09c;
}
a:visited {
color: #07a;
}
a:hover {
background-color: #eee;
}
#Header {
margin: 50px 0px 10px 0px;
padding: 17px 0px 0px 20px;
/* For IE5/Win's benefit height = [correct height] + [top padding] + [top and bottom border widths] */
height: 33px; /* 14px + 17px + 2px = 33px */
border-style: solid;
border-color: black;
border-width: 1px 0px; /* top and bottom borders: 1px; left and right borders: 0px */
line-height: 11px;
background-color: #eee;
/* Here is the ugly brilliant hack that protects IE5/Win from its own stupidity.
Thanks to Tantek Celik for the hack and to Eric Costello for publicizing it.
IE5/Win incorrectly parses the "\"}"" value, prematurely closing the style
declaration. The incorrect IE5/Win value is above, while the correct value is
below. See http://glish.com/css/hacks.asp for details. */
voice-family: "\"}\"";
voice-family: inherit;
height: 14px; /* the correct height */
}
/* I've heard this called the "be nice to Opera 5" rule. Basically, it feeds correct
length values to user agents that exhibit the parsing error exploited above yet get
the CSS box model right and understand the CSS2 parent-child selector. ALWAYS include
a "be nice to Opera 5" rule every time you use the Tantek Celik hack (above). */
body > #Header {
height: 14px;
}
#Content {
margin: 50px 50px 50px 200px;
padding: 10px;
}
#Menu {
position: absolute;
top: 100px;
left: 20px;
width: 172px;
padding: 10px;
background-color: #eee;
border: 1px dashed #999;
line-height: 17px;
/* Again, the ugly brilliant hack. */
voice-family: "\"}\"";
voice-family: inherit;
width: 150px;
}
/* Again, "be nice to Opera 5". */
body > #Menu {
width: 150px;
}
</style>
</head>
<body>
<div id="container" style="width:1000px">
<div id="header">
<p>
I have trained my classifier to separate wolves from sheep. Let's say sheep is a postive result and wolf is a negative result (sorry wolves). I now need to test it on my test set. This consists of <span class="TKAdjustableNumber" data-min="2" data-max="500" data-var="testWolves"> wolves </span>
and <span class="TKAdjustableNumber" data-min="2" data-max="500" data-var="testSheep"> sheep </span>
That&#8217s <b data-var="testSubjects"> </b> test subjects altogether.
</p>
<p>
Say my classifier correctly identifies <span class="TKAdjustableNumber" data-min="2" data-max="500 " data-var="TP"> sheep </span> as sheep (true positives)
and <span class="TKAdjustableNumber" data-min="2" data-max="500" data-var="TN"> wolves </span> as wolves (true negatives)
This gives us the confusion matrix below:
</p>
</div>
<div id="diagrams" style="height:800px;width:500px;float:right;">
<canvas id="confusion" width="500" height="500" style="border:1px solid #c3c3c3;">
Your browser does not support the HTML5 canvas tag.
</canvas>
</div>
<div id="description" style="height:800;width:400px;float:left;">
<p>
Now some statistics that need to be untangled!
</p>
<p>
<b>Precision</b> (aka positive predictive value) is TP/(TP+FP). In our case <b data-var="TP"></b>/(<b data-var="TP"></b> + <b data-var="FP"></b>) = <b data-var="precision" data-format="p2"> </b>
</p>
<p>
<b>Recall</b> (aka sensitivity, power) is TP/(TP+FN). In our case <b data-var="TP"></b>/(<b data-var="TP"></b> + <b data-var="FN"></b>) = <b data-var="recall" data-format="p2"> </b>
</p>
<p>
<b>Specificity</b> is TN/(TN+FP). In our case <b data-var="TN"></b>/(<b data-var="TN"></b> + <b data-var="FP"></b>) = <b data-var="specificity" data-format="p2"> </b>
</p>
<p>
<b>Negative</b> predictive value is TN/(TN+FN). In our case <b data-var="TN"></b>/(<b data-var="TN"></b> + <b data-var="FN"></b>) = <b data-var="npv" data-format="p2"> </b>
</p>
<p>
<b>Accuracy</b> is (TP+TN)/(TP+TN+FP+FN). In our case (<b data-var="TP"></b> + <b data-var="TN"></b> )/(<b data-var="TP"></b> + <b data-var="TN"></b>+<b data-var="FP"></b> + <b data-var="FN"></b>) = <b data-var="accuracy" data-format="p2"> </b>
</p>
<p>
<b>False discovery rate</b> is FP/(FP+TP). In our case <b data-var="FP"></b>/(<b data-var="FP"></b> + <b data-var="TP"></b>) = <b data-var="fdr" data-format="p2"> </b>
</p>
<p>
<b>False positive rate</b> (aka false alarm rate, fall-out) is FP/(FP+TN). In our case <b data-var="FP"></b>/(<b data-var="FP"></b> + <b data-var="TN"></b>) = <b data-var="fpr" data-format="p2"> </b>
</p>
</div>
</div>
<script>
//Function for confusion matrix
function drawConfusion(x, y, tp, tn, fp, fn) {
var c = document.getElementById("confusion");
var ctx = c.getContext("2d");
ctx.clearRect(0, 0, 500, 250);
//Draw box
ctx.beginPath();
ctx.rect(x, y, 100, 100);
ctx.moveTo(x, y + 50);
ctx.lineTo(x + 100, y + 50);
ctx.moveTo(x + 50, y);
ctx.lineTo(x + 50, y + 100);
//Add values
ctx.font = '14pt Calibri';
ctx.fillText("The Confusion Matrix", x-30, y -70);
ctx.fillText(tp, x + 15, y + 33);
ctx.fillText(tn, x + 65, y + 83);
ctx.fillText(fp, x + 15, y + 83);
ctx.fillText(fn, x + 65, y + 33);
//Add labels
ctx.font = '12pt Calibri';
ctx.fillText('Predicted', x + 20, y - 35);
ctx.fillText('Actual', x - 75, y + 52);
ctx.font = '18pt Calibri';
ctx.fillText('+', x + 16, y - 10);
ctx.fillText('-', x + 66, y - 10);
ctx.fillText('+', x - 25, y + 33);
ctx.fillText('-', x - 25, y + 83);
ctx.stroke();
};
function drawCircleChart(x, y, precision, recall, specificity, npv, accuracy, fdr, fpr) {
var c = document.getElementById("confusion");
var ctx = c.getContext("2d");
ctx.strokeStyle='pink';
ctx.clearRect(x - 20, y - 20, 400, 200);
ctx.beginPath();
//Calc y co-ords
//Add bars
for (var i=0;i<7;i++)
{
ctx.moveTo(x+20+50*i, y+150);
ctx.lineTo(x+20+50*i, y);
//ctx.stroke();
}
ctx.moveTo(x, y);
ctx.lineTo(x+340, y);
ctx.moveTo(x, y+150);
ctx.lineTo(x+340, y+150);
ctx.stroke();
//Not working because variable are out of scope
var yPR = y + (1 - precision) * 150;
var yRec = y + (1 - recall) * 150;
var ySpec = y + (1 - specificity) * 150;
var yNPV = y + (1 - npv) * 150;
var yAcc = y + (1 - accuracy) * 150;
var yFDR = y + (1 - fdr) * 150;
var yFPR = y + (1 - fpr) * 150;
ctx.moveTo(x + 35, yAcc);
ctx.arc(x + 20, yAcc, 15, 0, 2 * Math.PI);
ctx.moveTo(x + 85, yPR);
ctx.arc(x + 70, yPR, 15, 0, 2 * Math.PI);
ctx.moveTo(x + 135, yRec);
ctx.arc(x + 120, yRec, 15, 0, 2 * Math.PI);
ctx.moveTo(x + 185, ySpec);
ctx.arc(x + 170, ySpec, 15, 0, 2 * Math.PI);
ctx.moveTo(x + 235, yNPV);
ctx.arc(x + 220, yNPV, 15, 0, 2 * Math.PI);
ctx.moveTo(x + 285, yFDR);
ctx.arc(x + 270, yFDR, 15, 0, 2 * Math.PI);
ctx.moveTo(x + 335, yFPR);
ctx.arc(x + 320, yFPR, 15, 0, 2 * Math.PI);
ctx.fillStyle = 'pink';
ctx.fill();
ctx.fillStyle = 'black';
ctx.font = '11pt Calibri';
ctx.fillText("Acc", x + 9, 5 + yAcc);
ctx.fillText("Pr", x + 62, 5 + yPR);
ctx.fillText("Rec", x + 109, 5 + yRec);
ctx.fillText("SP", x + 159, 5 + ySpec);
ctx.fillText("NPV", x + 209, 5 + yNPV);
ctx.fillText("FDR", x + 259, 5 + yFDR);
ctx.fillText("FPR", x + 309, 5 + yFPR);
ctx.stroke();
ctx.strokeStyle='black';
};
//var c = document.getElementById("confusion");
//var ctx = c.getContext("2d");
//ctx.font = '12pt Calibri';
//ctx.fillText(' Predicted ', 150, 125);
//ctx.font = '20pt Calibri';
//ctx.fillText(' + -', 150, 145);
//ctx.stroke();
//Main tangle
var tangle = new Tangle(document.getElementById("container"), {
initialize : function() {
this.testWolves = 50;
this.testSheep = 100;
this.testSubjects = 150;
this.TN = 20;
this.TP = 60;
this.FP = 30;
this.FN = 40;
this.precision = 60 / (60 + 30);
this.recall = 60 / (60 + 40);
this.specificity = 20 / (20 + 30);
this.npv = 20 / (20 + 40);
this.accuracy = (20 + 60) / 150;
this.fdr = 30 / (30 + 60);
this.fpr = 30 / (30 + 20);
},
update : function() {
//Update all statistics
this.testSubjects = this.testWolves + this.testSheep;
this.FP = this.testWolves - this.TN;
this.FN = this.testSheep - this.TP;
this.precision = this.TP / (this.TP + this.FP);
this.recall = this.TP / (this.TP + this.FN);
this.specificity = this.TN / (this.TN + this.FP);
this.npv = this.TN / (this.TN + this.FN);
this.accuracy = (this.TN + this.TP) / this.testSubjects;
this.fdr = this.FP / (this.FP + this.TP);
this.fpr = this.FP / (this.FP + this.TN);
//Update circle chart
if (this.TP<=this.testSheep & this.TN<=this.testWolves){
drawCircleChart(80, 300, this.precision, this.recall, this.specificity, this.npv, this.accuracy, this.fdr, this.fpr);
drawConfusion(200, 120, this.TP, this.TN, this.FP, this.FN);
}
else
{
var c = document.getElementById("confusion");
var ctx = c.getContext("2d");
ctx.clearRect(0,0,500,500);
ctx.fillText("Exceeded number in test set!", 50,100);
}
//var c = document.getElementById("confusion");
//var ctx = c.getContext("2d");
//ctx.clearRect(150, 150, 80, 80);
//ctx.rect(150, 150, 80, 80);
//ctx.moveTo(150, 190);
//ctx.lineTo(230, 190);
//ctx.moveTo(190, 150);
//ctx.lineTo(190, 230);
//ctx.font = '12pt Calibri';
//ctx.fillText(this.TP, 170, 180);
//ctx.stroke();
}
});
</script>
</div>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment