Calculate entropy R2 for poLCA model

polca_entropy_r2.R

# MIT license
# Author: Daniel Oberski
# Input: result of a poLCA model fit
# Output: entropy R^2 statistic (Vermunt & Magidson, 2013, p. 71)
# See: daob.nl/wp-content/uploads/2015/07/ESRA-course-slides.pdf
# And: https://www.statisticalinnovations.com/wp-content/uploads/LGtecnical.pdf


machine_tolerance <- sqrt(.Machine$double.eps)
entropy.R2 <- function(fit) {
  entropy <- function(p) {
    p <- p[p > machine_tolerance] # since Lim_{p->0} p log(p) = 0
    sum(-p * log(p))
  }
  error_prior <- entropy(fit$P) # Class proportions
  error_post <- mean(apply(fit$posterior, 1, entropy))
  R2_entropy <- (error_prior - error_post) / error_prior
  R2_entropy
}

A few people have been writing me to point out that the code I shared in a slide here was wrong. It did not acccount for 0 and 1 posteriors in calculating entropy. This code assumes p*log(p) = 0) to deal with that issue. It is meant to work with the poLCA package in R for latent class analysis.

Code is provided as-is, without warranties, under the MIT license.