Skip to content

Instantly share code, notes, and snippets.

Avatar
🍓

hannah cushman garland hancush

🍓
View GitHub Profile
View Makefile
$(patsubst %, pensions_%.reordered.csv, $(shell seq 2012 1 2017)) : data/raw/pensions_2012-2017.csv
# some other operations on the downloaded/inflated pensions data
data/raw/pensions_%.csv : data/raw/pensions_%.tar
cd $(dir $@) && tar xvfz $(notdir $<)
data/raw/pensions_%.tar :
wget --no-use-server-timestamps \
https://bga-pensions-database.s3.amazonaws.com/raw/$(notdir $@) -O $@
View filter-dedupe-response.py
response = requests.post(some_post_data)
response_data = json.loads(response.data.decode('utf-8'))
print(response_data)
# {
# "status": "ok",
# "matches": [
# {
View iterchildren.py
from lxml import etree
html = '''
<h3>Offense: A Tale of Two Halves</h3>
<p>For a while there, it seemed the Bears' offense might never show up in London. The unit generated just two first downs and 44 total yards in the first half. Then <strong>Chase Daniel</strong> s
tarted hitting his spots, aided with some stellar work by <strong>Allen Robinson</strong> and <strong>Anthony Miller</strong>, to get the rally going. Unfortunately, it was too little, too late wi
th the defense not playing at its best.</p>
<h5>By the numbers:</h5>
<ul>
<li><strong>194 PASSING YARDS:</strong> Daniel had some good moments in completing 22 of 30 passes for 231 yards, but <a href="https://chicago.suntimes.com/bears/2019/10/6/20901945/daniel-takes-th
View duplicate-audit.csv
Employer ID
City of Chicago 16
Chicago 16
Metropolitan Pier and Exposition Authority 53
Metropolitan Pier Exposition Authority 53
Third Lake 364
Village Of Third Lake 364
Fox Valley Park District 519
Fox Valley Park District 519
Homewood Flossmoor Park District 541
View interval.py
# usage: python interval.py /path/to/your/file.csv
import csv
import sys
import re
def days_from_interval(interval):
if interval == 'never':
return None
View data-making-task.md

🤖 Make some data

Your mission

Write a Makefile that gets data from Data.gov and creates a CSV of the 50 most recent datasets uploaded. For each dataset, the output should include:

  • The name of the dataset
  • The date it was uploaded
  • A download link
View django_q.py
from django.db.models import Q
condition = Q()
employers = Employer.objects.all()
if params:
if params.get('name'):
employers = employers.annotate(search_vector=SearchVector('name'))
View extract-tables.sh
TABLES=$(mdb-tables /path/to/YOUR_ACCDB.accdb); \
for TABLE in $TABLES; \
do mdb-export /path/to/YOUR_ACCDB.accdb $TABLE > /path/to/$TABLE.csv; \
done;
@hancush
hancush / django-store-files-s3.md
Last active Mar 7, 2018
store django model-related files in s3
View django-store-files-s3.md

Install django-storages:

pip install django-storages

Add to settings:

INSTALLED_APPS = (
View cool-chart.pmd
# header
here's what this is
<<>>=
import pandas as pd
@
<<>>=
data = pd.read_csv('path/to/your/data') # returns a dataframe