Singularity basically pulls docker images and dumps them to disk as directories.
In testing, I discovered some things:
- It seems that when the Dockerfile doesn't have a
CMDentry at the end, it just dumps the filesystem contents. If you have aCMD, it dumps a binary.
I created a simple go image called simple-go.
The image builds a single binary, simple-go, which prints hello world.
I pushed it to my docker registry at docker.io/lcrown/simple-go, then pulled it on the HPC.
It correctly outputted hello world.
# create the project directory
mkdir simple-go
cd simple-go
# initialize the project
go mod init github.com/lcrownover/simple-go
mkdir -p cmd/simple-go/main.go
cat << EOF > cmd/simple-go/main.go
package main
import (
"fmt"
)
func main() {
fmt.Println("hello world")
}
EOF
# write the dockerfile per the instructions on the go container:
# https://hub.docker.com/_/golang
cat << EOF > Dockerfile
FROM golang:1.19
WORKDIR /usr/src/app
COPY go.mod ./
RUN go mod download && go mod verify
COPY . .
RUN go build -v -o /usr/local/bin/simple-go ./...
CMD ["simple-go"]
EOF
# build the image
#docker build -t lcrown/simple-go .
# build the image for amd64 since i'm on arm64
docker buildx build --platform linux/amd64 --load -t lcrown/simple-go:v1 .
# then push it to my registry
docker push lcrown/simple-go:v1
# switch to HPC
# build my container from my registry
singularity pull docker://lcrown/simple-go:v1
# now I can just run the .sif file
./simple-go_v1.sif # outputs "hello world"
Now, let's create a container with a bunch of python dependencies:
- requests
- pandas
- otter-grader
This is a super common use case, as we want to be able to freeze the versions of each of these dependencies.
mkdir python-deps-test
cd python-deps-test
# write the main script that will by run by the container
cat << EOF > main.py
#!/usr/bin/env python3
import requests
import pandas as pd
print("success")
EOF
# dependencies
virtualenv venv
source venv/bin/activate
pip install requests pandas otter-grader
pip freeze > requirements.txt
# write the dockerfile
cat << EOF > Dockerfile
FROM python:bullseye
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
COPY main.py /app/main.py
CMD ["python3", "/app/main.py"]
EOF
# build the image
#docker build -t lcrown/python-deps-test .
# build the image for amd64 since i'm on arm64
docker buildx build --platform linux/amd64 --load -t lcrown/python-deps-test .
# then push it to my registry
docker push lcrown/python-deps-test
# switch to HPC
# build my container from my registry
singularity build python-deps-test/ docker://lcrown/python-deps-test
# run the .sif
./python-deps-test_v1.sif # outputs "success"
TBD