Skip to content

Instantly share code, notes, and snippets.

@nmpowell
nmpowell / qs.sh
Last active August 29, 2015 13:57
Summarise Sun Grid Engine (SGE) qstat contents for the current user (and save output to a file)
function qs() {
# Also use qs as shortcut to qstat -j
if [ $# -gt 0 ]
then
jobNum=$1
qstat -j ${jobNum}
fi
# Just one call to qstat; "double quotes" around "${qstat_contents}" below maintain the line breaks
@nmpowell
nmpowell / qda.sh
Created March 25, 2014 23:37
Function to delete all Sun Grid Engine (SGE) jobs with a given string in their name
function qda() {
# Delete all jobs containing $1 in their (shortened) name
jobName=$1
# Get list of job numbers
jobList=$(echo "`qstat`" | grep "${jobName}" | awk '{print $1;}')
# Delete individually
for job in ${jobList}
do
qdel ${job}
done
@nmpowell
nmpowell / word_frequency.py
Last active December 22, 2022 06:23
Count words in a text file, sort by frequency, and generate a histogram of the top N
#!/usr/bin/python
"""Python script to create a histogram of words in a text file.
Usage: python word_frequency.py -f "/path/to/file.txt" -n 200
Specify the path to the text file as above. Manually specify the top N words to report (default 100).
Text file can contain punctuation, new lines, etc., but special characters aren't handled well.
@nmpowell
nmpowell / qsub_script.sh
Last active July 22, 2016 14:48
Some Sun Grid Engine shortcuts to include in .bashrc
# .bashrc
# simple qsub: assings 8GB memory for 1 hour. Call with: $ nq8 script.sh
alias nq8='qsub -l h_rt=01:00:00 -l tmem=8.0G -l h_vmem=8.0G -l vf=4.0 -l s_stack=10240 -R y -j y -S /bin/csh -b y -cwd -V'
# Shell function to echo the current time along with status updates
echoStatusTime() {
STATUS_TEXT=$1
NOW=$(date +%T)
DATE=$(date +%F)
@nmpowell
nmpowell / matlab_np.sh
Last active July 22, 2016 14:39
Bash / shell script for running Matlab jobs with the Sun Grid Engine. Qsub this whole script, rather than qsubbing Matlab directly.
#!/bin/sh
# matlab_np.sh
# Shell script to run a Matlab function from the bash shell
# Ensure Matlab function is on the path first
# NB Matlab breaks the bash shell, hence stty echo
# stty echo works by itself; stty sane is just-in-case; haven't confirmed that it's necessary.
#
# Example usage: $ matlab_np.sh functionName 'string arg' arg2 ${arg3}...
# Use with Qsub: $ ${QSUB_CMD} matlab_np.sh functionName arguments
# NB do not Qsub matlab directly as your arguments won't be passed to the Matlab function
@nmpowell
nmpowell / np.ahk
Created July 24, 2016 15:05
Some Autohotkey shortcuts
; Some Autohotkey shortcuts I use
#NoEnv
SetWorkingDir %A_ScriptDir%
GroupAdd, Explore, ahk_class CabinetWClass
GroupAdd, Explore, ahk_class ExploreWClass
SetTitleMatchMode, 2
; load many useful global variables
LoadVariables()
@nmpowell
nmpowell / windows_task_scheduler.py
Last active April 23, 2024 19:00
Python script to interact with existing Windows Task Scheduler tasks.
"""
Python script to interact with existing Windows Task Scheduler tasks.
CLI usage:
python windows_task_scheduler.py {enable|disable|run} -t "TaskName"
import usage:
import windows_task_scheduler as wts
wts.enable_task(task_name='TaskName')
wts.disable_task(task_name='TaskName')
{
"version": 1,
"disable_existing_loggers": "False",
"formatters": {
"simple": {
"format": "# {asctime} {name:8s} {levelname:6s} {message}",
"datefmt": "%m/%d/%Y %I:%M:%S %p",
"style": "{"
}
},
@nmpowell
nmpowell / check_list_contents.py
Last active May 25, 2017 08:41
Python snippets to check list contents
# Python snippets to check list contents
# Given two lists, A and B ...
# Check all the items from A are present in B:
set(B).issuperset(set(A))
# Given a list of substrings A and another list of (longer) strings B which might contain those substrings ...
# e.g.
A = ['one', 'two', 'three']
@nmpowell
nmpowell / csv_pandas_parse_emptylines.py
Last active July 1, 2021 16:15
Pandas functions to read .CSV files before / after an empty line is encountered.
# Python functions to read .CSV files into a Pandas DataFrame when the data of interest is before / after one or more empty/blank lines.
def csv_after_emptylines(filepath, bl_group_n=1, dtype=str):
""" Read a .CSV into a Pandas DataFrame, but only after at least one blank line has been skipped.
bl_group_n is the expected number of distinct blocks of blank lines (of any number of rows each) to skip before reading data.
NB: E.g. pd.read_csv(filepath, skiprows=[0, 1, 2]) works if you know the number of rows to be skipped. Use this function if you have a variable / unknown number of filled rows (to be skipped / ignored) before the empty rows.
"""
with open(filepath, newline='') as f: