Automatic Differentiation to calculation score function with Julia

forwarddiff-score-function.jl

# Automatic Differentiation example with forward accumulation
# Here we calculate the gradient of the loglikelihood of a normal
# distribution wrt to the mean and variance (i.e. the score function)

# load package for AD and Distributions
using ForwardDiff, Distributions

# parameters
μ = 10.0; # true mean
σ² = 1; # true variance
n = 100; # number of observations to be sampled

# put parameters in an array
x = [μ, σ²];

# sample observations
y = rand(Normal(μ, √σ²), n);

# create function to take the gradient of
h(x) = loglikelihood(Normal(x[1], √x[2]), y)

# calculate gradient and evaluate at truth
ForwardDiff.gradient(h, [μ, σ²])

# analytically derived score functions for comparison
# (note that sigma is assumed 1)

# partial log L / partial μ
sum(y - μ)

# partial log L / partial σ²
-n / 2 + 0.5 * sum((y-μ).^2)