Skip to content

Instantly share code, notes, and snippets.

Avatar

Johannes Buchner JohannesBuchner

View GitHub Profile
@JohannesBuchner
JohannesBuchner / joss_make_latex.sh
Created Jan 23, 2021
Make LaTeX and PDF from JOSS markdown papers
View joss_make_latex.sh
#!/bin/bash
# you need to install:
# pip install openbases
# sudo apt install texlive-xetex pandoc pandoc-citeproc
PDF_INFILE=paper.md
PDF_LOGO=logo.png
PDF_OUTFILE=paper.pdf
TEX_OUTFILE=paper.tex
@JohannesBuchner
JohannesBuchner / rsync_on_change.sh
Created Dec 15, 2015
rsync loop that updates remote directory when local directory changes (using inotify)
View rsync_on_change.sh
while true
do
rsync -avz ./ user@host:remote/directory/
inotifywait -r ./
done
View benchmark.sh
# columns: id1,id2,id3,id4,id5,id6,v1,v2,v3
f=G1_1e7_1e2_0_0.csv
awk="time mawk"
# groupby simple
$awk -F, 'NR>1 { a[$1] += $7 } END {for (i in a) print i, a[i]}' $f >/dev/null
$awk -F, 'NR>1 { a[$1,$2] += $7 } END { for (comb in a) { split(comb,sep,SUBSEP); print sep[1], sep[2], a[sep[1],sep[2]]; }}' $f >/dev/null
$awk -F, 'NR>1 { a[$3] += $7; n[$3]++; b[$3] += $9; } END {for (i in a) print i, a[i], b[i]/n[i];}' $f >/dev/null
$awk -F, 'NR>1 { a[$4] += $7; n[$4]++; b[$4] += $8; } END {for (i in a) print i, a[i]/n[i], b[i]/n[i];}' $f >/dev/null
@JohannesBuchner
JohannesBuchner / xray_opt_gif.sh
Created Dec 18, 2020
Make a gif flipping between an X-ray and optical image at some coordinate
View xray_opt_gif.sh
#!/bin/bash
# example usage:
# bash xray_opt_gif.sh 155.87737 +19.86508
RA=$1
DEC=$2
wget -nc "https://alasky.unistra.fr/hips-thumbnails/thumbnail?ra=${RA}&dec=${DEC}&fov=0.21750486127986932&width=500&height=500&hips_kw=CDS%2FP%2FSDSS9%2Fcolor" -O opt.jpg
wget -nc "https://alasky.unistra.fr/hips-thumbnails/thumbnail?ra=${RA}&dec=${DEC}&fov=0.21750486127986932&width=500&height=500&hips_kw=xcatdb%2FP%2FXMM%2FPN%2Fcolor" -O xmm.jpg
@JohannesBuchner
JohannesBuchner / rsyncprogress.py
Last active Sep 1, 2020
Progress bar for rsync
View rsyncprogress.py
"""
Progress bar for rsync
========================
Shows file progress and total progress as a progress bar.
Usage
---------
Run rsync with -P and pipe into this program. Example::
@JohannesBuchner
JohannesBuchner / cachestan.py
Created May 18, 2020
Build and cache Stan models smartly (ignoring changes in comments and white spaces)
View cachestan.py
import re
import pystan
import hashlib
import pickle
import os
def build_model(code):
lines = code.split("\n")
lines = [re.sub('//.*$', '', line).strip() for line in lines]
lines = [line.replace(' ', ' ').replace(' ', ' ').replace(' ', ' ')
@JohannesBuchner
JohannesBuchner / run_processes_when_idle.sh
Created May 12, 2020
Stop/resume processes when the user is active/inactive in Linux/Xorg
View run_processes_when_idle.sh
#!/bin/bash
# how to identify the processes to STOP/CONT
# this is any part of process command line
PROCESS="myscript.py"
# run this in one terminal:
# it tries to resume the processes every so often
while sleep 10m; do
pgrep -f "${PROCESS}"|xargs -rt kill -CONT
@JohannesBuchner
JohannesBuchner / poissonerr.py
Last active Aug 12, 2019
Obtain correct error bars from observed counts (fraction in sample, detected events, etc.)
View poissonerr.py
"""
If you ever made a plot of "fractions" or "rates" with symmetric error
bars, like in the plot shown, this mini-tutorial is for you. Here is how
to compute correct error bars, so that uncertainties in the fractions
do not go below 0% or above 100%.
If you have a histogram for
instance, and you detected k objects in a given bin. What is the rate
underlying at quantiles q=0.1, 0.5(median), 0.9?
"""
@JohannesBuchner
JohannesBuchner / mcmc.py
Created Aug 5, 2019
Minimalistic MCMC implementation
View mcmc.py
import numpy as np
def mcmc(logfunction, x0, nsteps, sigma_p):
samples = np.empty((nsteps, len(x0)))
logL0 = logfunction(x0)
naccepts = 0
for i in range(nsteps):
x1 = np.random.normal(x0, sigma_p)
logL1 = logfunction(x1)
if logL1 - logL0 > np.log(np.random.uniform()):
@JohannesBuchner
JohannesBuchner / toh5.py
Created Jun 2, 2019
Translate a csv/npy/csv.gz integer file to efficiently compressed HDF5
View toh5.py
import sys
import numpy
import h5py
filename = sys.argv[1]
outfilename = filename.replace('.npy', '').replace('.gz', '').replace('.csv', '') + '.h5'
if filename.endswith('.npy'):
print('loading NPY...')
data = numpy.load(filename)
else: