gistfile1.md

Fyro

Fyro Models

An Fyro model is an F# definition of the form:

[<ReflectedDefinition>]
let model (param1, .... paramN) = <distribution-expression>

Note: parameters must be tupled not curried Note: not pattern matching can be used in the parameters

A is of the form:

   <dist-expr> =
     |  let v = sample <dist-expr>               // sample
     |  let v = <value-expr>                             // bind
     |  <ident> (<value-expr>, ..., <value-expr>)  // primitve, ident is a primitive distribution 
     |  <ident> (<value-expr>, ..., <value-expr>)  //  sub-model, ident is a model  
     |  if <value-expr> then <dist-expr> else <dist-expr>   //  Constraint: must define same model parts on each?
     |  <value-expr>                                     // return 

     // Possible addition: returning sub-structure of models
     |  {| <ident> = <value-expr> ... |}                 // return, model structure 
     
   <value-expr> = 
     | <value-expr>  + <value-expr>  // arithmetic
     | float-constant
     | <ident>

Fyro Inference Specifications

Fyro models require an inference specification before inference can be performed. The nature of the inference specification will depend on the backend inference process being used.

Discussion: Having an internal IR and its library of operations???

It is possible to have an internal IR

   // Internal IR
   type FyroDistExpr =
      | LetSample of ....
      | SubModel of string * FyroValueExpr list
      | ...
     
   type FyroValueExpr =
      | Add of ....
      | Ident of string

And a possible library of operations:

   let GetIR (q:  Expr) : FyroDistExpr =
       ...
       
   let Sample (inp: FyroDistExpr) : Tensor = ....
   
   let MeanVariationalLogPdf (inp: FyroDistExpr) (variationParameters: Map<string,Tensor>) : Tensor = 
       ...

   MeanVariationalLogPdf (GetIR <@ model @>) (map [ "a", tensor(....); "b", tensor(....)  ])

Examples

[<ReflectedDefinition>]
let model x = 
   x + x

Examples that are not allowed

let model x = 
   normal x 0.0 //???

Discussion: Naming elements of your model ????

Many operations on the internal IR require naming structure of the model. One approach is to use strings like "m.a" systematically derived from the expansion of the model.

Another is to have each model always publish its structure as, say, an anonymous record in the return position.

    [<ReflectedDefinition>]
    let model mu = 
        let a = sample (normal (0.0, 1.0))
        let b = sample (normal (mu, 1.0))
        // Possible addition: returning sub-structure of models
        {| a = a; b = b |}
        
    let m = model 1.0
    m.a
    m.b

Discussion: conditions in distribution expressions

Are conditionals allowed in distribution expressions?

[<ReflectedDefinition>]
let model x = 
   if x > 3.0 then 
      let v = sample (normal(x, 0.0))
      v
   else
      let v = sample (bernoulli(x, 0.0))
      v

Defining new primitive distributions

A primitive distribution is defined by giving its mapping to DiffSharp sampling primitives:

[<ReflectedDefinition>]
let normal (x,y) = DiffSharp.Distributions.Normal(x,y).sample()

[<ReflectedDefinition>]
let bernoulli (x,y) = DiffSharp.Distributions.Bernoulli(x,y).sample()

You can define an entirely new DiffSharp distribution like this

Topic: making a sysematic type distinction between distributions and values

As an aside, some F# programming with distributions makes a systematic type distinction between distribution values and samples, using F# computation expression syntax as neat syntax for building distributions.

Fyro is not doing this as yet.

val dist : DistributionBuilder //  builder for Distribution<_> values

val normal : float -> float -> Distribution<float>

// F# syntax with naive F# executable semantics given by 'dist'
[<ReflectedDefinition>]
let modelA () : Distribution<Tensor> = 
    dist { 
        let! a = normal 0.0 1.0
        let! b = normal a 1.0
        return b 
    }