guillermo-carrasco

import re
from collections import Counter
from datetime import datetime

import emoji
import pandas as pd

class WhatsappChat:
    DATE_FORMAT = "[%m/%d/%y, %H:%M:%S]"

guillermo-carrasco

import argparse
import logging
import re
import subprocess
import sys
from pathlib import Path
from typing import List


def fetch_latest_version(library):

guillermo-carrasco

import glob
import os
import random

if not os.path.exists('selected.txt'):
    open('selected.txt', 'a').close()

with open('selected.txt', 'r') as f:
    selected = f.read().splitlines()

guillermo-carrasco

import yaml
import logging
import logging.config

with open('config.yaml', 'r') as f:
    conf = yaml.load(f)

logging.config.dictConfig(conf)

myapp = logging.getLogger('myapp')

guillermo-carrasco

import logging

parent_log = logging.getLogger('parent')
child_1 = logging.getLogger('parent.child_1')
child_2 = logging.getLogger('parent.child_2')

guillermo-carrasco

version: 1
formatters:
    custom_format:
        format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
filters:
    myapp:
        name: myapp
handlers:
    console:
        class: logging.StreamHandler

guillermo-carrasco

In this step we'll create a swarm cluster from two nodes. You should've received access to two nodes in AWS.

1. SSH in to the first node: ssh -i docker_demo.pem ubuntu@IP1
2. Figure out the internal IP address of the node: ifconfig and check the eth0 address (10.0....)
3. Initialize the Swarm cluster: `docker swarm init --listen-addr INTERNAL_IP --advertise-addr INTERNAL_IP
4. Open a new tab and SSH to the other node: ssh -i docker_demo.pem ubuntu@IP2
5. Check the internal IP of that node: ifconfig and eth0
6. Join to the cluster with the command the previous command printed out. Before executing the command, you need to the extend it with the --listen-addr and --advertise-addr parameters (using the internal IP of the second node): docker swarm join TOKEN --listen-addr 10.0.... --advertise-addr 10.0..... node1...
7. Get back to the first node and verify the results: docker node ls

guillermo-carrasco

#!/bin/bash

# Trim the data to the first 1.000.000 reads (4.000.000 lines)
zcat synthetic_challenge_set3_normal_NGv3_1.fq.gz | head -n 4000000 > synthetic_challenge_set3_normal_NGv3_1_trimmed.fq && gzip synthetic_challenge_set3_normal_NGv3_1_trimmed.fq
zcat synthetic_challenge_set3_normal_NGv3_2.fq.gz | head -n 4000000 > synthetic_challenge_set3_normal_NGv3_2_trimmed.fq && gzip synthetic_challenge_set3_normal_NGv3_2_trimmed.fq
zcat synthetic_challenge_set3_tumor_NGv3_1.fq.gz | head -n 4000000 > synthetic_challenge_set3_tumor_NGv3_1_trimmed.fq && gzip synthetic_challenge_set3_tumor_NGv3_1_trimmed.fq
zcat synthetic_challenge_set3_tumor_NGv3_2.fq.gz | head -n 4000000 > synthetic_challenge_set3_tumor_NGv3_2_trimmed.fq && gzip synthetic_challenge_set3_tumor_NGv3_2_trimmed.fq

guillermo-carrasco

#!/usr/bin/env python

import re

if __name__=="__main__":

    with open('stepic_dataset.txt', 'r') as f:
        dna = f.readline().rstrip()
        k = int(f.readline().rstrip())

guillermo-carrasco

import logging
from logbook.queues import RedisHandler

h = RedisHandler(key='python_agent')
l = logging.getLogger('redis')
l.addHandler(h)

l.warn('hejjjjjj')