Skip to content

Instantly share code, notes, and snippets.

🤓
data says that...

Guillermo Carrasco guillermo-carrasco

🤓
data says that...
Block or report user

Report or block guillermo-carrasco

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View logging.py
import yaml
import logging
import logging.config
with open('config.yaml', 'r') as f:
conf = yaml.load(f)
logging.config.dictConfig(conf)
myapp = logging.getLogger('myapp')
View descendant.py
import logging
parent_log = logging.getLogger('parent')
child_1 = logging.getLogger('parent.child_1')
child_2 = logging.getLogger('parent.child_2')
View config.yaml
version: 1
formatters:
custom_format:
format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
filters:
myapp:
name: myapp
handlers:
console:
class: logging.StreamHandler
View Step 1 | Create cluster.md

In this step we'll create a swarm cluster from two nodes. You should've received access to two nodes in AWS.

  1. SSH in to the first node: ssh -i docker_demo.pem ubuntu@IP1
  2. Figure out the internal IP address of the node: ifconfig and check the eth0 address (10.0....)
  3. Initialize the Swarm cluster: `docker swarm init --listen-addr INTERNAL_IP --advertise-addr INTERNAL_IP
  4. Open a new tab and SSH to the other node: ssh -i docker_demo.pem ubuntu@IP2
  5. Check the internal IP of that node: ifconfig and eth0
  6. Join to the cluster with the command the previous command printed out. Before executing the command, you need to the extend it with the --listen-addr and --advertise-addr parameters (using the internal IP of the second node): docker swarm join TOKEN --listen-addr 10.0.... --advertise-addr 10.0..... node1...
  7. Get back to the first node and verify the results: docker node ls
@guillermo-carrasco
guillermo-carrasco / trim_data.sh
Last active Oct 13, 2015
Small scrip to trim data for cancer/normal pipeline testing in AWS
View trim_data.sh
#!/bin/bash
# Trim the data to the first 1.000.000 reads (4.000.000 lines)
zcat synthetic_challenge_set3_normal_NGv3_1.fq.gz | head -n 4000000 > synthetic_challenge_set3_normal_NGv3_1_trimmed.fq && gzip synthetic_challenge_set3_normal_NGv3_1_trimmed.fq
zcat synthetic_challenge_set3_normal_NGv3_2.fq.gz | head -n 4000000 > synthetic_challenge_set3_normal_NGv3_2_trimmed.fq && gzip synthetic_challenge_set3_normal_NGv3_2_trimmed.fq
zcat synthetic_challenge_set3_tumor_NGv3_1.fq.gz | head -n 4000000 > synthetic_challenge_set3_tumor_NGv3_1_trimmed.fq && gzip synthetic_challenge_set3_tumor_NGv3_1_trimmed.fq
zcat synthetic_challenge_set3_tumor_NGv3_2.fq.gz | head -n 4000000 > synthetic_challenge_set3_tumor_NGv3_2_trimmed.fq && gzip synthetic_challenge_set3_tumor_NGv3_2_trimmed.fq
View Shell provisioning warnings
==> default: Running provisioner: shell...
default: Running: /var/folders/7t/bwttrszn4dv6zr1kyy2lq98m0000gn/T/vagrant-shell20150924-44091-1gp277n.sh
==> default: stdin: is not a tty
==> default: discarding /home/vagrant/miniconda/bin from PATH
==> default: prepending /home/vagrant/miniconda/envs/prod/bin to PATH
==> default: /vagrant/scout/ext/backend/config_parser.py:95: Warning: Click detected the use of the unicode_literals __future__ import. This is heavily discouraged because it can introduce subtle bugs in your code. You should instead use explicit u"" literals for your unicode strings. For more information see http://click.pocoo.org/python3/
==> default: type=click.File('w')
==> default: /home/vagrant/miniconda/envs/prod/lib/python2.7/site-packages/vcf_parser/parser.py:265: Warning: Click detected the use of the unicode_literals __future__ import. This is heavily discouraged because it can introduce subtle bugs in your code. You should instead use explicit u"" literals for your unicode strin
@guillermo-carrasco
guillermo-carrasco / gist:6826648
Created Oct 4, 2013
NAIVE implementation of the problem "Find the most common k-mer in a DNA sequence
View gist:6826648
#!/usr/bin/env python
import re
if __name__=="__main__":
with open('stepic_dataset.txt', 'r') as f:
dna = f.readline().rstrip()
k = int(f.readline().rstrip())
View gist:6623249
import logging
from logbook.queues import RedisHandler
h = RedisHandler(key='python_agent')
l = logging.getLogger('redis')
l.addHandler(h)
l.warn('hejjjjjj')
@guillermo-carrasco
guillermo-carrasco / deinterleave.c
Created May 18, 2013
FASTQ files deinterleaver.
View deinterleave.c
/*
Deinterleave FASQ files containing both reads from a paired-end read.
The script makes the following assumption:
* FASTQ format (4 lines per read), and the two reads (fordward and reverse) in a row.
USAGE: :/deinterleave file.fastq f.file.fastq r.file.fastq
RESULT: f.file.fastq r.file.fastq containing fordward and backward reads, respectively
Can deinterleave FASTQ files of any size, does not consume memory. The execution time is
I/O bounded.
@guillermo-carrasco
guillermo-carrasco / simNGS_unique_ids.patch
Created May 13, 2013
Unique read IDs for simLibrary
View simNGS_unique_ids.patch
diff -ru simNGS.orig/src/simlibrary.c simNGS/src/simlibrary.c
--- simNGS.orig/src/simlibrary.c 2013-05-13 17:45:17.000000000 +0200
+++ simNGS/src/simlibrary.c 2013-05-13 18:06:41.000000000 +0200
@@ -33,12 +33,13 @@
#define QUOTE(A) Q_(A)
#define DEFAULT_COV 0.055
#define DEFAULT_OUT "fasta"
+#define DEFAULT_UNIQUE "not"
#define DEFAULT_INSERT 400
#define DEFAULT_NCYCLE 45
You can’t perform that action at this time.