ltrainpr

Data Engineer's Responsibilities (not all encompassing):

• Building data platforms
• Define data architecture and data modeling
• Handle data in various formats
• Create ETL or ELT pipelines as well as streaming data pipelines
• Schedule and deploy pipelines
• Build frameworks or code for data management activities
• Make data accessible with right governance in place
• Enable self service access to data

cccntu

❯ rm out.csv
❯ cat 1.py
from glob import glob
import mmap

files = glob("data/*")

files.sort(key=lambda x: int(x.split("/")[-1].split(".")[0]))
write_f = open("out.csv", "w+b")

robinsones

Helpful Papers on A/B Testing

• Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained
• Online Controlled Experiments at Large Scale
• Seven Pitfalls to Avoid in Experiments
• 12 A/B Split Testing Mistakes
• 3 Top Mistakes that Invalidate A/B Test Results
• Seven Rules of Thumb for Web Site Experiments
• Online Experimentation at Microsoft
• [Online Controlled Experiments and A/B Tests](http://www.exp-platform.com/Documents/2015%20Online%20Controlled%20Experiments_EncyclopediaOfMLDM.pd

wesm

import gc
import os
import time

import numpy as np
import pandas as pd
from pyarrow.compat import guid
import pyarrow as pa
import pyarrow.parquet as pq
import snappy

nfaggian

from __future__ import print_function

import multiprocessing
import ctypes
import numpy as np

def shared_array(shape):
    """
    Form a shared memory numpy array.
    

dfm

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import division, print_function
import emcee
import numpy as np
import matplotlib.pyplot as pl

np.random.seed(123)

disnet

defaults write -app Skim SKAutoReloadFileUpdate -boolean true

uasi

require 'formula'

class Vim < Formula
  homepage 'http://www.vim.org/'
  url 'ftp://ftp.vim.org/pub/vim/unix/vim-7.3.tar.bz2'
  head 'https://vim.googlecode.com/hg/'
  sha256 '5c5d5d6e07f1bbc49b6fe3906ff8a7e39b049928b68195b38e3e3d347100221d'
  version '7.3.682'

  def features; %w(tiny small normal big huge) end