Skip to content

Instantly share code, notes, and snippets.

View szs8's full-sized avatar

S szs8

10 followers · 14 following
  • New York, New York
View GitHub Profile
@szs8
szs8 / Effective_Engineer.md
Created December 29, 2017 13:20 — forked from rondy/Effective_Engineer.md

Effective Engineer - Notes

What's an Effective Engineer?

  • They are the people who get things done. Effective Engineers produce results.

Adopt the Right Mindsets

@szs8
szs8 / to_redshift.py
Created June 18, 2016 01:13 — forked from ajsmith007/to_redshift.py
# see also https://github.com/wrobstory/pgshift
import gzip
from io import StringIO, BytesIO
from functools import wraps
import boto
from sqlalchemy import MetaData
from pandas import DataFrame
from pandas.io.sql import SQLTable, pandasSQL_builder
@szs8
szs8 / DataFrameGUI.py
Created June 9, 2016 12:58 — forked from jsexauer/DataFrameGUI.py
A sophisticated GUI to interact with DataFrame objects
"""
DataFrameTable
==============
Quick and Dirty Qt app to view pandas DataFrames. Includes sorting and
filterting.
Based on qtpandas in pandas sandbox module, by Jev Kuznetsov
Usage:
@szs8
szs8 / anaconda openssl
Created June 7, 2013 14:55
Fixing openssl errors on OS X with anaconda python distribution
OS X ships with an ancient version of openssl and I ran into problems using starcluster with Anaconda python distribution (APD).
Every starcluster command failed with the following error:
SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
The continuum guys are working on fix. Meanwhile a quick workaround is to use homebrew to build python with a newer openssl version and then substitute the _ssl.so in APD with the homebrew _ssl.so
1) brew install python --with-brewed-openssl
@szs8
szs8 / yahoo
Created May 19, 2013 18:18
Downloading splits and dividends from yahoo
#!/usr/bin/env python
import dateutil
from datetime import date, datetime
import numpy as np
import pandas as pd
import requests
class YahooException(Exception):
@szs8
szs8 / aj.py
Created September 9, 2012 18:20
asof join in pandas
def diffCols(df1, df2):
""" Find columns in df1 not present in df2
Return df1.columns - df2.columns maintaining the order which the resulting
columns appears in df1.
Parameters:
----------
df1 : pandas dataframe object
df2 : pandas dataframe objct
Pandas already offers df1.columns - df2.columns, but unfortunately
@szs8
szs8 / pandasgrid.py
Created September 7, 2012 21:12
Display a pandas dataframe in a wx grid
#!/usr/bin/env python
import wx
from wx import EVT_CLOSE
import wx.grid as gridlib
EVEN_ROW_COLOUR = '#CCE6FF'
GRID_LINE_COLOUR = '#ccc'
class PandasTable(wx.Frame):
@szs8
szs8 / quote_price_levels.py
Created September 7, 2012 19:44
marking quote price levels in pandas dataframe
In [25]: quotes.head()
Out[25]:
bid ask bsize asize
2012-09-06 09:30:00.026000 13.34 13.44 3 16
2012-09-06 09:30:00.043000 13.34 13.44 3 17
2012-09-06 09:30:00.121000 13.36 13.65 1 10
2012-09-06 09:30:00.386000 13.36 13.52 21 1
2012-09-06 09:30:00.440000 13.40 13.44 15 17
In [26]: quotes["mid"] = 0.5 * (quotes.bid + quotes.ask)
@szs8
szs8 / pandas_dbms.py
Created September 7, 2012 17:59 — forked from catawbasam/pandas_dbms.py
Python PANDAS : load and save Dataframes to sqlite, MySQL, Oracle, Postgres
# -*- coding: utf-8 -*-
"""
example use of pandas with oracle mysql postgresql sqlite
lightly tested.
to do:
save/restore index (how to check table existence? just do select count(*)?),
finish odbc,
add booleans?,
sql_server?
@szs8
szs8 / latency.txt
Created May 31, 2012 13:36 — forked from jboner/latency.txt
Latency numbers every programmer should know
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns
Mutex lock/unlock 25 ns
Main memory reference 100 ns
Compress 1K bytes with Zippy 3,000 ns
Send 2K bytes over 1 Gbps network 20,000 ns
Read 1 MB sequentially from memory 250,000 ns
Round trip within same datacenter 500,000 ns
Disk seek 10,000,000 ns