Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
rawr

Vanessasaurus vsoch

💭
rawr
View GitHub Profile
@vsoch
vsoch / Singularity
Last active May 8, 2019
An example of setting a custom environment variable already defined in the container
View Singularity
Bootstrap: docker
From: centos:7
# sudo singularity build lamps.simg Singularity
%environment
now=`date '+%Y_%m_%d'`
hostname=`hostname`
APPNAME="lammps"
@vsoch
vsoch / output_file.md
Created May 4, 2019
Saving output to a file and printing to terminal
View output_file.md

This is executing a command to the container to list at the root (/) and then redirect stdout and stderr to "out.txt"

$ singularity exec salad_latest.sif ls / 2>&1 | tee out.txt
bin
code
dev
environment
etc
home
@vsoch
vsoch / analyze_ubuntu.md
Created May 3, 2019
Container Diff Examples
View analyze_ubuntu.md
$ container-diff analyze --type=apt ubuntu:16.04

-----Apt-----

Packages found in ubuntu:16.04:
NAME                        VERSION                              SIZE
-adduser                    3.113 nmu3ubuntu4                    648K
-apt                        1.2.31                               3.4M
-base-files                 9.4ubuntu4.8                         312K
@vsoch
vsoch / markdown-details.md
Last active May 1, 2019
An example of how to put a really long error log (or similar) in markdown! It's all about details. :)
View markdown-details.md

Lots of discussion, blardee boo, and here is my Dockerfile that is important to you!

FROM debian:stretch-backports

LABEL authors="@bnmnetp,@vsoch,@yarikoptic"

# docker build -t runstone/server .
@vsoch
vsoch / singularity-pull-examples.md
Last active Apr 25, 2019
All the ways you can pull a Singularity container from shub.
View singularity-pull-examples.md

Ways to Pull

How do you pull, my friend? Let me count the ways!

1. Pull by Tag

$ singularity pull shub://vsoch/singularity-images:latest
 35.30 MiB / 35.30 MiB [=========================================================================================================================================] 100.00% 4.07 MiB/s 8s
111
@vsoch
vsoch / package-investigation.md
Last active Apr 24, 2019
Singularity package investigation (again) reveals that self.stage.source_path is not reliable
View package-investigation.md

This is a debugging session for spack. The issue is that the self.stage.source_path changes (unexpectedly) in the middle of do_stage, and it's not clear why this happens. I'm at the HEAD of the repo, just pulled the fix to remove the empty line:

commit b92fe5f248892069674de2f090e4da879b5c83d4 (HEAD -> update/package-singularity, origin/update/package-singularity)
Author: Vanessa Sochat <vsochat@stanford.edu>
Date:   Wed Apr 24 09:58:08 2019 -0400

    removing blank link
    
    Signed-off-by: Vanessa Sochat <vsochat@stanford.edu>
@vsoch
vsoch / python-hashing.py
Created Feb 16, 2019
A quick way to run a single file through a bunch of hashing algorithms, via Python's hashlib
View python-hashing.py
import hashlib
filename = '/home/vanessa/.singularity/shub/vanessa-marshmallow-latest@8dc77d81e1775986b89f042112c5ee81.sif'
for algorithm in hashlib.algorithms_guaranteed:
hasher = getattr(hashlib, algorithm)()
with open(filename, "rb") as f:
for chunk in iter(lambda: f.read(4096), b""):
hasher.update(chunk)
try:
print("%s: %s" %(algorithm, hasher.hexdigest()))
except:
@vsoch
vsoch / singularity_example_bash.sh
Last active Feb 4, 2019
An example script to run a Singularity container for a user on the list
View singularity_example_bash.sh
#!/bin/bash
# Before you run this script, pull your container somewhere, and then provide
# the full path as an argument to this script
# If your $HOME is small, export to where you have more space like scratch
# export SINGULARITY_CACHEDIR=$SCRATCH/.singularity
# singularity pull --name antismash-standalone-4.2.0.sif docker://antismash/standalone:4.2.0
# Then provide the full container path as the last argument.
set -o errexit
@vsoch
vsoch / pull_static_registry.sh
Last active Feb 2, 2019
A quick example of how to pull a container from a static registry (https://singularityhub.github.io/sregistry-org)
View pull_static_registry.sh
# Define the registry base, the container URI, and the tag
REGISTRY=https://singularityhub.github.io/registry-org
CONTAINER=singularityhub/centos
TAG=latest
# Use curl and jq to parse the manifest and obtain the download link
STORAGE=$(curl "${REGISTRY}/${CONTAINER}/manifests/${TAG}/" | jq --raw-output '.layers [] .urls []')
DIGEST=$(curl "${REGISTRY}/${CONTAINER}/manifests/${TAG}/" | jq --raw-output '.layers [] .digest')
# Use Singularity to pull (here I'm using the Dockerized version, note it's pulling to $PWD)
@vsoch
vsoch / with-google-dns.sh
Created Jan 14, 2019
Singularity Hub Debugging
View with-google-dns.sh
## The following shows some debugging output when Google nameservers are added to /etc/resolv.conf (8.8.8.8 and 8.8.4.4)
#################################################################
$ whois singularity-hub.org
Domain Name: SINGULARITY-HUB.ORG
Registry Domain ID: D402200000000429751-LROR
Registrar WHOIS Server: whois.google.com
Registrar URL: http://domains.google.com
Updated Date: 2019-01-14T16:27:45Z