arthuralvim

Tópicos Importantes sobre Boas Ferramentas de ETL

1. Conectividade com os dados

Permitir que seja possível acoplar/conectar com qualquer fonte de dados seja ela um banco de dados relacional, não relacional, uma planilha ou um webservice.

2. Performance

Processos de Extração, Transformação e Carga costumam demorar bastante devido ao poder de processamento e I/O demandados.

arthuralvim

Check if $var is set using ! i.e. check if expr is false

[ ! -z "$var" ] || echo "Empty" [ ! -z "$var" ] && echo "Not empty" || echo "Empty"

[[ ! -z "$var" ]] || echo "Empty" [[ ! -z "$var" ]] && echo "Not empty" || echo "Empty"

====== The classic test command ======

arthuralvim

# docker

alias doco='docker-compose'
alias doma='docker-machine'
alias doenv='printenv | grep DOCKER'
alias doim='docker images'
alias dorm='docker rm'
alias dohosts='doma ls --format "{{.Name}} {{.Active}}"'
alias domals='doma ls'
alias dormi='docker rmi'

arthuralvim

from flask import Flask
import peewee
from flask.ext import admin
from flask.ext.admin.contrib.peewee import ModelView

app = Flask(__name__)
app.config['SECRET_KEY'] = '123456790'
db = peewee.SqliteDatabase('test.sqlite', check_same_thread=False)

class BaseModel(peewee.Model):

arthuralvim

# -*- coding: utf-8 -*-

import datetime
from pytz import timezone
from pytz import utc

fmt = '%d/%m/%Y %H:%M:%S %Z'

date_utc_now = datetime.datetime.utcnow()

arthuralvim

# Create a reference folder with some data
mkdir foo
echo "hey" > foo/A.txt
echo "scm" > foo/B.txt
echo "git" > foo/C.txt

# Generate the list of hashes for each file
# -b = bare mode (strips any leading directory
#      information from displayed filenames)
md5deep -b foo/* > hashes.txt

arthuralvim

# Inside the directory that needs to be hashed
# recursive, show estimated time, relative paths
md5deep -rle -W /path/to/result.md5 *

# List files with the same hash, printing the hash-path of each file (remember to keep at least one of them)
sort result.md5 | uniq -D -w 32

# Count the number of duplicate files, path to first encountered file with that hash is shown only | sort by frequency
sort result.md5 | uniq -cd -w 32 | sort -nr

arthuralvim

echo "backend: TkAgg" >> ~/.matplotlib/matplotlibrc

arthuralvim

"""IPython startup script to detect and inject VIRTUAL_ENV's site-packages dirs.

IPython can detect virtualenv's path and injects it's site-packages dirs into sys.path.
But it can go wrong if IPython's python version differs from VIRTUAL_ENV's.

This module fixes it looking for the actual directories. We use only old stdlib
resources so it can work with as many Python versions as possible.

References:
http://stackoverflow.com/a/30650831/443564

arthuralvim

#Use this URL to get your client authorized for Instafeed.JS plugin for Instagram.

1. First login into your Instargam account
2. Then go to https://www.instagram.com/developer/
3. At the top click on the Button that says "Manage Clients".
4. If you have not Register a new client.
5. Fill out everything and where it says redirect url put this url: http://instafeedjs.com
6. Then on the security tab make sure you uncheck "Disable implicit OAuth"
7. Finally use this link to authoruize Instafeed. Where it says "[Client ID]" replace that including the brackets with your clientID from instagram:
8. https://instagram.com/oauth/authorize/?client_id=[Client ID]&redirect_uri=http://instafeedjs.com&response_type=token