Skip to content

Instantly share code, notes, and snippets.

Avatar
🏠
Working from home

Roman Valls Guimera brainstorm

🏠
Working from home
View GitHub Profile
@brainstorm
brainstorm / tasks.py
Created Mar 4, 2011
Celery tasks test
View tasks.py
#!/usr/bin/env python
import subprocess
from celery.task import task
# Task overlapping:
# lvm_create --> archive --> transfer(delayed?) --> (verify) --> lvm_destroy
# basecalling --> fastq transfer (8h ?)
# Without LVM:
# archive --> transfer --> verify
@brainstorm
brainstorm / logbook_mailhandler.py
Created Apr 13, 2011
python logbook mailhandler
View logbook_mailhandler.py
#!/usr/bin/env python
# Small logbook snippet addressing some confusion on record injection
import logbook
handler = logbook.MailHandler("test@example.com", ['recipient@example.com'],
format_string=u'''Subject: [BCBB] {record.extra[run]} \n\n {record.message}''', level='INFO', bubble = True)
log = logbook.Logger("mailtest.log")
@brainstorm
brainstorm / add_final_a_index.py
Created Sep 19, 2011
Illumina hiseq 2000 sequencers add an extra base "A" at the end of each QSEQ read
View add_final_a_index.py
#!/usr/bin/env python
import csv
import sys
import os
ss_reader = csv.DictReader(open(sys.argv[1], 'rU'))
ss_writer = csv.DictWriter(open(os.path.splitext(sys.argv[1])[0]+"_a.csv", 'w'), ss_reader.fieldnames)
ss_writer.writerow(dict((fn,fn) for fn in ss_reader.fieldnames))
@brainstorm
brainstorm / tarit_hiseq2000.sh
Created Sep 19, 2011
md5 + tar for illumina hiseq 2000 datasets, with extra metadata tar
View tarit_hiseq2000.sh
#!/bin/sh
# Generates a TAR archive while computing MD5 checksum for each file.
# From VeriTAR: http://www.g-loaded.eu/2007/12/01/veritar-verify-checksums-of-files-within-a-tar-archive/
DATASET_PATH=`readlink -f $1`
DATASET=`basename $DATASET_PATH`
EXCLUDE_FILES="--exclude .DS_Store --exclude .AppleDouble"
echo "Creating TAR + MD5 for dataset: $DATASET"
tar -cvpf $DATASET.tar $DATASET_PATH $EXCLUDE_FILES \
@brainstorm
brainstorm / dotfilter_symlinker.py
Created Feb 15, 2012
Try to recover illumina dataset with missing .filter files
View dotfilter_symlinker.py
#!/bin/env python
import os
import logging
import glob
import random
from collections import *
fters = defaultdict(set)
all = set(["%04d.filter" % x for x in range(1, 70)])
@brainstorm
brainstorm / quick_delivery.sh
Created Feb 28, 2012
Cronify simple data transfer for simple data delivery
View quick_delivery.sh
#!/bin/sh
DATASET="
SRC=""
DST=""
if [ -d ${SRC}${DATASET} ]; then
echo "Copying ${SRC}${DATASET} into $DST..."
chown -R $USER:uppmax ${SRC}${DATASET}
rsync -a ${SRC}${DATASET} ${DST}
@brainstorm
brainstorm / slack_backup.py
Last active Oct 1, 2015
Saves daily chat backup for future reference in the context of a jupyter lab notebook
View slack_backup.py
import os
import time
import json
import gzip
import datetime
import requests
import cPickle as pickle
from datetime import datetime
slack_token = "<SLACK_TOKEN>"
@brainstorm
brainstorm / timelapse.py
Created Oct 15, 2012
Timelapse experiment with cheap hardware (efikamx and benq old (web)camera)
View timelapse.py
#!/usr/bin/env python
import flickrapi
import datetime
import os
import shutil
import subprocess
api_key = 'SETME!!!'
api_secret = 'SETME!!!'
@brainstorm
brainstorm / svg_ipython.svg
Last active Nov 17, 2015
IPython SVG and pyveplot broken graph
View svg_ipython.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View pouta-docker-machine.sh
#!/bin/bash
# To use an Openstack cloud you need to authenticate against keystone, which
# returns a **Token** and **Service Catalog**. The catalog contains the
# endpoint for all services the user/tenant has access to - including nova,
# glance, keystone, swift.
#
# *NOTE*: Using the 2.0 *auth api* does not mean that compute api is 2.0. We
# will use the 1.1 *compute api*
export OS_AUTH_URL=https://pouta.csc.fi:5001/v2.0