claczny/installing_singularity_on_macbook_pro_m1_pro.md

## installing_singularity_on_macbook_pro_m1_pro.md

      
    Raw
  

              installing_singularity_on_macbook_pro_m1_pro.md
            
          
    I wanted to install Singularity on my M1 Pro (Ventura 13.1).
As this chip is arm64 based, it turned out rather tricky.
Some resources that I consulted although none of them presented a self-contained solution are below.
I hadn't tried the solution via UTM as I had access to a Parallels Desktop license and did not necessarily want to have yet-another-tool installed.

Instructions from Singularity itself: https://docs.sylabs.io/guides/3.0/user-guide/installation.html#install-on-windows-or-mac
Video using UTM:
https://www.youtube.com/watch?v=aKHbaytlS00
https://github.com/manuparra/singularitycontainers-on-m1-arm64
Another set of general instructions.: https://github.com/sPHENIX-Collaboration/Singularity/blob/master/OSX_installationguide.md

But now for the actual steps that worked:

Let's get a virtual machine up-and-running:

# Install Parallels Desktop. N.B. You need a license for that.
# There is NO need to install Parallels' Linux, e.g., Ubuntu 22.04 ARM64 directly via Parallels Desktop, s.a., 
https://kb.parallels.com/128445 and "List of supported distributions" link at 
https://www.parallels.com/products/desktop/resources/#requirements-guestos
# Go to your Terminal, e.g., iTerm2
# Install vagrant via homebrew
brew install --cask vagrant
brew install --cask vagrant-manager
# Install the vagrant plugins for Parallels Desktop
vagrant plugin install vagrant-parallels
cd $HOME
mkdir virtual_machines
cd virtual_machines
mkdir singularity-vm
cd singularity-vm
# `vagrant init singularityware/singularity-2.4` does NOT work as the box is not M1/arm64 compatible
vagrant init jeffnoxon/ubuntu-20.04-arm64 # Works on M1/arm64
vagrant up # Download vagrant box and setup the virtual machine
vagrant ssh # Basically login to your virtual machine, i.e., you are good to go on the command line by now.


Getting the software environment ready

# These commands are executed *within* your virtual machine
sudo apt update # Update package list and versions of your virtual machine
sudo apt upgrade # Perform upgrade of outdated packages; for a list of outdated packages, run `
apt list --upgradable`
# Install dependencies:
sudo apt-get update && sudo apt-get install -y build-essential libssl-dev uuid-dev libgpgme11-dev squashfs-tools libseccomp-dev pkg-config libglib2.0-dev
export VERSION=1.19.5 OS=linux ARCH=arm64 && \
    wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
    tar -xzvf go$VERSION.$OS-$ARCH.tar.gz && \ # Docs say to unpack to /usr/local (`-C /usr/local`) but the script to install GO dep (see below) has issues resolving paths then. I prefer to keep it as much in the user space as possible anyways.
    rm go$VERSION.$OS-$ARCH.tar.gz
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    source ~/.bashrc
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh # Following instructions at
https://golang.github.io/dep/docs/installation.html#binary-installation


Finally install Singularity; from sources

# Here, I had to adjust the path from the official Singularity documentation. It is no longer `singularity` but `singularity-ce`
export VERSION=3.10.5 && # adjust this as necessary \
    mkdir -p $GOPATH/src/github.com/sylabs && \
    cd $GOPATH/src/github.com/sylabs && \
    wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-ce-${VERSION}.tar.gz && \
    tar -xzf singularity-ce-${VERSION}.tar.gz && \ 
    cd ./singularity-ce-${VERSION} && \
    ./mconfig

make -C ./builddir && \
sudo make -C ./builddir install #  The `sudo` seems needed for the install. Everything else seems to be fine without it.

. /usr/local/etc/bash_completion.d/singularity # Enable bash completion. Not sure if this survives a restart or needs to be added to shell config, e.g., $HOME/.bashrc

singularity version # Check if singularity works

Testing the installation

Mini test for building yourself

Basically it means building your own container as it seems that arm64 containers are not yet very many.
The instructions are taken from https://github.com/manuparra/singularitycontainers-on-m1-arm64
cd $HOME
git clone https://github.com/manuparra/singularitycontainers-on-m1-arm64.git
cd singularitycontainers-on-m1-arm64
sudo singularity build lolcow.sif lolcow.def # Build the container, thereby generating the `lolcow.sif` file
singularity run lolcow.sif 

Test fetching a container useful for machine learning

singularity pull arm64-docker-pytorch-tensorflow.sif docker://sonoisa/deep-learning-coding:pytorch1.12.0_tensorflow2.9.1
singularity exec arm64-docker-pytorch-tensorflow.sif  python -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))" # Example output but note that this will vary due to the tf.random: `tf.Tensor(-145.93863, shape=(), dtype=float32)`

Other possibly useful links


Vagrant CLI cheatsheet: https://gist.github.com/wpscholar/a49594e2e2b918f4d0c4
For Intel Macs and using vagrant with Parallels Desktop: https://kb.parallels.com/en/122843