Bayesian inference

xholonWorkbook.xml

<?xml version="1.0" encoding="UTF-8"?>
<!--Xholon Workbook http://www.primordion.com/Xholon/gwt/ MIT License, Copyright (C) Ken Webb, Tue May 06 2014 11:29:15 GMT-0400 (EDT)-->
<XholonWorkbook>

<Notes><![CDATA[
Xholon
------
Title: Bayesian inference
Description: 
Url: http://www.primordion.com/Xholon/gwt/
InternalName: ceb4083319ebfa4a7bd0
Keywords: 

My Notes
--------
In this workbook I explore how to use Bayesian inference in a Xholon app.

The wikipedia page[1] begins with:
`In statistics, Bayesian inference is a method of inference in which Bayes' rule is used to update the probability estimate for a hypothesis as additional evidence is acquired.`
This and the rest of the descriptive text doesn't work for me as an introduction.
I learn best by working through examples, and the page does include several simple examples that I can implement using Xholon.
Once I've worked through the practical examples I should be able to better understand the more technical content of the page.
I'll first do the "Probability of a hypothesis" example, which is about Fred and two bowls of cookies.
`Suppose there are two full bowls of cookies. Bowl #1 has 10 chocolate chip and 30 plain cookies, while bowl #2 has 20 of each. Our friend Fred picks a bowl at random, and then picks a cookie at random. We may assume there is no reason to believe Fred treats one bowl differently from another, likewise for the cookies. The cookie turns out to be a plain one. How probable is it that Fred picked it out of bowl #1?`

Bayes' Rule according to [1]
             P(E|H) * P(H)
    P(H|E) = -------------
                  P(E)

Bayes' Rule in the "Probability of a hypothesis" example
                       P(E|H1) * P(H1)
    P(H1|E) = ---------------------------------
              P(E|H1) * P(H1) + P(E|H2) * P(H2)

References
----------
(1) http://en.wikipedia.org/wiki/Bayesian_inference
(2) http://en.wikipedia.org/w/index.php?title=Bayesian_inference&oldid=605608761
    permalink
(3) http://en.wikipedia.org/wiki/Bayes%27_rule
(4) http://en.wikipedia.org/wiki/Bayes%27_theorem
(5) http://jim-stone.staff.shef.ac.uk/BookBayes2012/bookbayesch01.pdf

]]></Notes> 

<_-.XholonClass>
  <BayesianSystem/>
  
  
  <!-- types of objects (entities) in the statistician Thomas Bayes' domain [1] -->
  <BayesianEntity>
    
    <HypothesisBE/> <!-- H "H stands for any hypothesis whose probability may be affected by data (called evidence below). Often there are competing hypotheses, from which one chooses the most probable." -->
    
    <EvidenceBE/> <!-- E data "the evidence E corresponds to new data that were not used in computing the prior probability" -->
    
    <AntecedentBE>
      <PriorProbabilityBE/> <!-- P(H) "the prior probability, is the probability of H before E is observed. This indicates one's previous estimate of the probability that a hypothesis is true, before gaining the current evidence." -->
      
      <LikelihoodBE/> <!-- P(E|H) "the probability of observing E given H. As a function of E with H fixed, this is the likelihood. The likelihood function should not be confused with P(H | E) as a function of H rather than of E. It indicates the compatibility of the evidence with the given hypothesis." -->
    </AntecedentBE>
    
    <PosteriorProbabilityBE/> <!-- P(H|E) "the posterior probability, is the probability of H given E, i.e., after E is observed. This tells us what we want to know: the probability of a hypothesis given the observed evidence." -->
    
    <MarginalLikelihoodBE/> <!-- P(E) "P(E) is sometimes termed the marginal likelihood or 'model evidence'. This factor is the same for all possible hypotheses being considered. (This can be seen by the fact that the hypothesis H does not appear anywhere in the symbol, unlike for all the other factors.) This means that this factor does not enter into determining the relative probabilities of different hypotheses." -->
    
  </BayesianEntity>
  
  
  <!-- types of objects in our hungry friend Fred's domain -->
  <Bowl/>
  <Cookie>
    <ChocolateChipCookie/>
    <PlainCookie/>
  </Cookie>
  
  
  <!-- types of people -->
  <Person>
    <HungryFriend/>
    <Statistician/>
  </Person>
  
</_-.XholonClass>

<xholonClassDetails>
  
</xholonClassDetails>

<BayesianSystem>
  <Bowl roleName="#1">
    <ChocolateChipCookie multiplicity="10"/>
    <PlainCookie multiplicity="30"/>
  </Bowl>
  <Bowl roleName="#2">
    <ChocolateChipCookie multiplicity="20"/>
    <PlainCookie multiplicity="20"/>
  </Bowl>
  
  <HungryFriend roleName="Fred"/>
  
  <Statistician roleName="Thomas Bayes"/>
</BayesianSystem>

<HungryFriendbehavior implName="org.primordion.xholon.base.Behavior_gwtjs"><![CDATA[
  var fred;
  var beh = {
    postConfigure: function() {
      fred = this.cnode.parent();
      fred.println("I'm Fred. I look after the cookie bowls.");
    },
    act: function() {
      // Fred picks a bowl at random (XPath is one-based so possible bowl numbers are 1 and 2)
      var bowlNum = Math.ceil(Math.random() * 2);
      var bowl = fred.xpath("../Bowl[" + bowlNum + "]");
      var result = "bowl" + bowlNum + " " + bowl.name();
      // Fred picks a cookie at random
      var cookieNum = Math.ceil(Math.random() * 40);
      var cookie = bowl.xpath("Cookie[" + cookieNum + "]");
      result = result + " cookie" + cookieNum + " " + cookie.name();
      fred.println(result);
    }
  }
]]></HungryFriendbehavior>

<Statisticianbehavior implName="org.primordion.xholon.base.Behavior_gwtjs"><![CDATA[
  var thomas;
  var beh = {
    postConfigure: function() {
      thomas = this.cnode.parent();
      thomas.println("I'm Thomas Bayes. I was an English statistician, philosopher and Presbyterian minister.");
      this.calc();
    },
    act: function() {
      
    },
    calc: function() {
      var likelihood1 = 30 / 40;
      var priorProbability1 = 0.5;
      var likelihood2 = 20 / 40;
      var priorProbability2 = 1.0 - priorProbability1;
      var one = likelihood1 * priorProbability1;
      var two = likelihood2 * priorProbability2;
      var posteriorProbability1 = one / (one + two);
      thomas.println("Probability that Fred picked the plain cookie out of bowl #1 = " + posteriorProbability1);
    }
  }
]]></Statisticianbehavior>

<SvgClient><Attribute_String roleName="svgUri"><![CDATA[data:image/svg+xml,
<svg width="100" height="50" xmlns="http://www.w3.org/2000/svg">
  <g>
    <title>Bowl</title>
    <rect id="BayesianSystem/Bowl[1]" fill="#98FB98" height="50" width="50" x="25" y="0"/>
    <g>
      <title>Cookie</title>
      <rect id="BayesianSystem/Bowl[1]/Cookie" fill="#6AB06A" height="50" width="10" x="80" y="0"/>
    </g>
  </g>
</svg>
]]></Attribute_String><Attribute_String roleName="setup">${MODELNAME_DEFAULT},${SVGURI_DEFAULT}</Attribute_String></SvgClient>

</XholonWorkbook>