Skip to content

Instantly share code, notes, and snippets.

Will Furnass willfurnass

Block or report user

Report or block willfurnass

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@willfurnass
willfurnass / export_data.py
Created Feb 20, 2019
Python snippet to export data to a file in a particular format
View export_data.py
import os
from typing import Dict
import numpy as np
def export_data(data: Dict[np.ndarray],
filename: str,
cols_per_line: int = 8) -> None:
"""Export dictionary of 1d float arrays to a file.
View keybase.md

Keybase proof

I hereby claim:

  • I am willfurnass on github.
  • I am willfurnass (https://keybase.io/willfurnass) on keybase.
  • I have a public key whose fingerprint is 5350 0720 33E5 2DBF 6C9E 9EB1 DEFE 4719 102C 352F

To claim this, I am signing this object:

@willfurnass
willfurnass / jsv_crlf_rejecter.tcl
Created Apr 9, 2018
Grid Engine JSV snippet to reject jobs containing CRLF
View jsv_crlf_rejecter.tcl
#!/usr/bin/tclsh
set sge_root $env(SGE_ROOT)
source "$sge_root/util/resources/jsv/jsv_include.tcl"
proc jsv_on_start {} {
jsv_send_env
}
proc lexists name {
@willfurnass
willfurnass / paramiko_test.py
Last active May 18, 2018
Submitting a job to a remote Grid Engine cluster using Paramiko (Python SSH client)
View paramiko_test.py
import tempfile
import os
import time
import paramiko
# Hostname of a 'submission host' in the Grid Engine cluster we want to submit a job to
hostname = 'sharc.shef.ac.uk'
# Username for logging in to the cluster via SSH
username = "te1st"
@willfurnass
willfurnass / dss2emails.sh
Last active Jan 2, 2018
Extract usernames and emails from a Synology DSS config dump file (.dss)
View dss2emails.sh
#!/bin/bash
# Will Furnass
# Oct 2017
if [[ $# -lt 1 ]]; then
echo 1>&2 "Extract list of user email addresses from Synology DSS '.dss' config dump"
exit 1
fi
dss_path="$1"
@willfurnass
willfurnass / README.md
Created Jun 1, 2017
Separating Reveal.js from your presentation content using git subrepo
View README.md

Initialise a git repository:

cd somedir
git init

Install git subrepo if you haven't done so already:

https://github.com/ingydotnet/git-subrepo#installation
@willfurnass
willfurnass / sge_cheatsheet.md
Last active Nov 21, 2019
(Son of) Grid Engine tips and tricks
View sge_cheatsheet.md

Random Grid Engine tips and tricks

The following work with Son of Grid Engine (SGE) 8.1.9 as configured on the University of Sheffield's ShARC and Iceberg clusters.

Jobs with dependancies

You can use the -hold_jid <<job-name or job-name>> option to make jobs run only when other jobs have finished, rather than having jobs start and sit waiting for other tasks to complete.

View tmux_cheatsheet.md

tmux tips

Default prefix: <ctrl>b

Session management

  • tmux new -s session_name - creates a new tmux session named session_name
  • tmux a - attach to first available session
  • tmux attach -t session_name - attaches to an existing tmux session named session_name
  • tmux switch -t session_name - switches to an existing session named session_name
@willfurnass
willfurnass / qrshx
Created Apr 19, 2017
Simple wrapper for Sun Grid Engine's qrsh that propagates SGE environment variables and enables X forwarding
View qrshx
#!/bin/sh
# David Jones and Will Furnass (University of Sheffield)
# `qrsh` is often the most convenient utility for starting interactive shell
# sessions on Sun/Son of Grid Engine (SGE) clusters but it has some limitations (as
# configured on the ShARC/Iceberg clusters):
#
# - none of the standard SGE environment vars (e.g. JOB_ID, PE_HOSTFILE,
# NSLOTS) that are defined in qsub/qsh batch/interactive sessions are set
# (annoying but not the end of the world);
@willfurnass
willfurnass / ruffus_simple_test.py
Created Jan 11, 2017
Simple test of the Ruffus workflow manager
View ruffus_simple_test.py
#!/usr/bin/env python
from __future__ import print_function
from ruffus import transform, suffix, pipeline_run
# A test of using Ruffus to locally run a very simple pipeline comprised of
# tasks defined as Python functions
# Can run this from a conda environnment created using
# conda create -n drmaatest -c bioconda python=2.7 ruffus drmaa
You can’t perform that action at this time.