vsoch

Quick Links

These are some quick links for the singularity work for Cromwell! Because slack + markdown == pain

• here is the container recipe
• docs for building and running
• the backend configuration with a struggle bus for me to get a PID for the running container :P
• singularity workflow opts (must build container locally since not on Shub!
• possible avenue to get singularity in singularity

vsoch

# This needs to already have been done
# install.packages("psych")
library(psych)

SCRATCH <- Sys.getenv("SCRATCH")
setwd(SCRATCH)

# This command assumes the file exists at /scratch/$USER/emotionzs.csv
alldata = read.csv("emotionzs.csv", header = FALSE)

vsoch

FROM ubuntu:16.04
# docker build -t oscar-test .
RUN apt-get update && \
    apt-get -y install git \
                   build-essential \
                   libtool \
                   squashfs-tools \
                   autotools-dev \
                   libarchive-dev \
                   automake \

vsoch

# Step 1: Find experiments
# =========================
docker run vanessa/expfactory-builder list

# The library is at https://expfactory.github.io/experiments
# You can contribute to it too! 
# --> https://expfactory.github.io/expfactory/contribute#the-experiment-repository

# Step 2: Generate a Dockerfile
# =============================

vsoch

alias nom=cd
alias tummy=git

tummy init breakfast 
nom breakfast
tummy checkout -b brunch
touch pancakes
tummy add pancakes
tummy commit -a -m 'brunch!'
tummy brunch

vsoch

# Test 1: Cannot Create Bucket

# Here is a set of commands to run to better explain the "Cannot get or create
# the group" error. I suspect the underlying issue is not having permissions to use storage.
# make sure these same variables are still exported

export GOOGLE_CLOUD_PROJECT=...
export GOOGLE_APPLICATION_CREDENTIALS="...

# You need to have storage admin permissions on this project

vsoch

Pull a container with wine

$ singularity pull shub://katakombi/dolmades
Progress |===================================| 100.0% 
Done. Container is at: /home/vanessa/katakombi-dolmades-master-latest.simg

Shell inside

vsoch

FROM vanessa/expfactory-builder:base

########################################
# Configure
########################################

ENV EXPFACTORY_STUDY_ID expfactory
ENV EXPFACTORY_SERVER localhost
ENV EXPFACTORY_CONTAINER true
ENV EXPFACTORY_DATA /scif/data

vsoch

import requests

response = requests.get('http://singularity-hub.org/api/container/vsoch/singularity-images/')

if respnse.status_code == 200:
    response.json()
    # {'branch': 'master',
    #  'commit': '130504089d5b2b44e2788992d0de75b625da6796',
    #  'id': 2688,
    #  'image': 'https://www.googleapis.com/download/storage/v1/b/singularityhub/o/singularityhub%2Fgithub.com%2Fvsoch%2Fsingularity-images%2F130504089d5b2b44e2788992d0de75b625da6796%2Fa1025471b564766d08bdf2cb062c795c%2Fa1025471b564766d08bdf2cb062c795c.simg?generation=1525030158750776&alt=media',

vsoch

# Here is an equivalent (interactive) version to parse with R, and two methods (using rjson and jsonlite) are shown.

install.packages('rjson')
install.packages('jsonlite')

# Load the package required to read JSON files.
library("rjson")
library("jsonlite")

# Give the input file name to the function.