dmbates

---
jupyter: julia-1.7
---

```{julia}
using RCall
```

```{julia}
RCall.ijulia_setdevice(MIME("image/svg+xml"))

dmbates

using CSV, DataFrames, Downloads, Tar

datadir = "biofast-data-v1"
tarball = "$datadir.tar.gz"
if !isfile(tarball)
  dataurl = joinpath(
    "https://github.com/lh3/biofast/releases/download",
    datadir,
    tarball,
  )

dmbates

dmbates

---
title: JAGS and lme4
author: Douglas Bates
date: March 28, 2014
output:
  html_document:
    toc: yes
    highlight: tango
    fig_width: 7
    fig_height: 5.5

dmbates

dmbates

Background

The current version of the (lme4)[https://github.com/lme4/lme4] package for (R)[http://www.r-project.org) is based on a rather questionable practice of storing external pointers from C++ objects in an R reference class object. The idea was that when the reference class object was created the corresponding C++ class instance would be created. All changes to this object would be made through the external pointer in the C++ code.

This is risky because any changes to the R objects in the reference class could cause changes in the location of its values, after which all bets are off because the R object and the C++ object are referring to different memory locations.

It turns out that this does happen in the development version of R, which will become R-3.1.0, if the LAZY_DUPLICATE_OK flag is set. The symptom is that deepcopy methods are not copying and I suspect this is because the lazy duplication doesn't duplicate before the external pointer is constructed. We were living outsid

dmbates

dmbates

Objectives

The dimensions of data on DNA variation such as single nucleotide polymorphisms or SNPs can be very large, involving thousands or millions of SNPs, measured on potentially thousands of individuals. Typical genotyping platforms may examine from 50K(K=thousand) to 2.5M (M= millions) SNPs. Some platforms could be even denser. There are 2 nucleotides (A, C, G or T) at each position (one on each chromosome). If the genotyping read is not sufficiently good, a missing value could be recorded in one or both chromosomes for that position/SNP. A frequently used re-codification of the nocleotide data is to replace the characters (i.e. alleles) by the count of the allele with the lower frequency in the sample, or according to a pre-specified allele as determined in the genotyping platform and software. Thus, instead of storing a pair of nucleotides (e.g., AA, AG, GG), researchers store the individual’s genotype as either 0,1,2, or NA. In thi

dmbates

Formula, ModelFrame and ModelMatrix types

In R, the functions for fitting statistical models based on a linear predictor usually allow for a formula/data specification of the model. The data part is a data.frame object in R. In Julia the corresponding type is a DataFrame as implemented in the DataFrames package.

A formula object in R is an expression whose top-level operator is a unary or binary tilde, ~. The interpretation of many operators, including +, *, ^, \ and : are overridden in the formula language. See the result of

?formula

in R for the details.

dmbates

Regular sparse matrices

In previous versions of the lme4 package for R, the random-effects model matrix, Z, or its transpose, Zt, have been stored as compressed sparse column-oriented matrices, which is a general sparse matrix representation. As implemented in the Matrix package for R, an m by n dgCMatrix matrix with nnz non-zeros has three vector slots: p, which is the length n + 1 vector of column pointers, i, which is the length nnz vector of row indices, and x, which is the length