Skip to content

Instantly share code, notes, and snippets.

View jrandall's full-sized avatar

Joshua C. Randall jrandall

41 followers · 12 following
View GitHub Profile
@jrandall
jrandall / pyproject.toml
Created February 19, 2020 18:49
[tool.poetry]
name = "project"
version = "0.1.0"
description = ""
authors = ["Example Author <example@example.com>"]
packages = [
{ include = "project" }
]
[tool.poetry.dependencies]
@jrandall
jrandall / crash.log
Created August 1, 2018 16:54
terraform-openstack-provider v.1.7.0 panic when using data "openstack_networking_network_v2" to get external network
2018/08/01 16:32:44 [INFO] Terraform version: 0.11.7 41e50bd32a8825a84535e353c3674af8ce799161
2018/08/01 16:32:44 [INFO] Go runtime version: go1.9
2018/08/01 16:32:44 [INFO] CLI args: []string{"/usr/local/bin/terraform", "plan"}
2018/08/01 16:32:44 [DEBUG] Attempting to open CLI config file: /home/xxxxxxx/.terraformrc
2018/08/01 16:32:44 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2018/08/01 16:32:44 [INFO] CLI command args: []string{"plan"}
2018/08/01 16:32:44 [INFO] command: empty terraform config, returning nil
2018/08/01 16:32:44 [DEBUG] command: no data state file found for backend config
2018/08/01 16:32:44 [DEBUG] New state was assigned lineage "a9872a3b-0002-40d8-a9f9-46d7fc56fcdc"
2018/08/01 16:32:44 [INFO] command: backend initialized: <nil>
@jrandall
jrandall / monitor-passenger-status-pool.sh
Created March 9, 2018 14:59
#!/bin/bash
watch "passenger-status --show=pool | grep 'Last used' | perl -p -e 's/^.*(Last[ ]*used[:].*[hms])[ ]*.*/\$1/g; s/(\d+)h/60*60*\$1/ge; s/(\d+)m/60*\$1/ge; s/(\d+)s/\$1/g; s/(\d+)[ ]*(\d+)?[ ]*(\d+)?/\$1+\$2+\$3/ge; s/(\d+)/\$1s ago/g;' | sort -k3g | uniq -c | awk 'BEGIN {FS=\" \"; OFS=\"\\t\";} {c=\$1; \$1=\"\"; print \"Last used \"\$4\" ago:\", gensub(\"0\", \"#\", \"g\", sprintf(\"%0\"c\"d (%s)\", 0, c));}'"
@jrandall
jrandall / metric-monitor.sh
Last active March 8, 2018 17:51
Monitor a metric resulting from a command run periodically
#!/bin/bash
period_s=$1
window_size=$2
target_metric=$3
metric_command=$4
half_window=$((${window_size} / 2))
metric=""
metrics=()
@jrandall
jrandall / tool-draft-3.cwl
Created December 20, 2017 17:42
#!/usr/bin/env cwl-runner
cwlVersion: draft-3
class: CommandLineTool
inputs:
- id: message
type: string
inputBinding:
position: 1
@jrandall
jrandall / irods_init.c
Created December 6, 2017 18:05
Demonstration of problems in iRODS 4.2.2 client plugins when run from RTLD_LOCAL context
#include <stdio.h>
#include <rodsClient.h>
static struct {
rcComm_t *conn;
rodsEnv env;
} irods = { NULL };
//extern "C" int irods_init();
@jrandall
jrandall / convert_ancestrydna_to_23_and_me_format.sh
Last active November 29, 2017 23:03
Simple awk script to convert the AncestryDNA data export format to one compatible with the 23andMe format.
#!/bin/bash
cat AncestryDNA.txt | awk 'BEGIN {OFS="\t";} $1~/^#/ {print} $1=="rsid" {print "#",$0} $1!~/^#/ && $1!="rsid" { if($4=="0") {$4="-"}; if($5=="0") {$5="-"}; if($2=="23") {$2="X"}; if($2=="24") {$2="Y"}; if($2=="25") {$2="X"}; if($2=="26") {$2="MT"}; print $1, $2, $3, $4$5;}' > AncestryDNA_23andme_format.txt
@jrandall
jrandall / Dockerfile
Created January 19, 2016 21:20
mercury/hgi-arvados-base dockerfile
FROM ubuntu:14.04
MAINTAINER jcrandall@alum.mit.edu
# Switch to root user for installation
USER root
# Install updated packages and prerequisites
RUN \
apt-get -q=2 update && \
apt-get -q=2 -y upgrade && \
@jrandall
jrandall / arvados-node-ping.sh
Last active January 15, 2016 17:22
Arvados node ping script (suitable for running from CRON) that updates node properties
#!/bin/bash
set -euf -o pipefail
ARVADOS_API_HOST="api.arvados.sanger.ac.uk"
SCRATCH_PATH="/data/crunch-tmp"
LOG_PATH="/var/log/arvados-node-ping.log"
UUID=$(grep \"uuid\" /root/node.json |cut -f4 -d\")
PING_SECRET=$(grep \"ping_secret\" /root/node.json |cut -f4 -d\")
@jrandall
jrandall / arvados-set-node-info.sh
Last active January 15, 2016 11:43
Set arvados node properties using `arv` cli and `jq`(note that these will be overwritten by node pings and can also be set from a ping)
for uuid in $(arv node list --select='["uuid","hostname"]' | jq -r '.items[] | select(.hostname | test("humgen-0[1234]")) | .uuid'); do arv node update --uuid ${uuid} --node "$(arv node get --uuid ${uuid} | jq 'with_entries(select(.key=="properties") | .value.total_cpu_cores=40 | .value.total_ram_mb=192 | .value.total_scratch_mb=1000000)')"; done
for uuid in $(arv node list --select='["uuid","hostname"]' | jq -r '.items[] | select(.hostname | test("humgen-05")) | .uuid'); do arv node update --uuid ${uuid} --node "$(arv node get --uuid ${uuid} | jq 'with_entries(select(.key=="properties") | .value.total_cpu_cores=32 | .value.total_ram_mb=256 | .value.total_scratch_mb=750000)')"; done