Skip to content

Instantly share code, notes, and snippets.

View hancush's full-sized avatar
🍓

hannah cushman garland hancush

🍓
View GitHub Profile
#!/bin/sh
youtube-dl -o "_%(id)s.%(ext)s" $1
infile=$(ls | head -n1)
palette="/tmp/palette.png"
filters="fps=12,scale=500:-1:flags=lanczos"
@hancush
hancush / README.md
Last active January 6, 2017 18:28
quick bar charts from the command line w agate

requirements

  • agate (pip install agate)

psst! if you're using csvkit, congrats! you've already got agate!

usage

to download make_bars.py, open a terminal to your project directory and run

@hancush
hancush / README.md
Last active May 17, 2017 17:59
drafts for making data 001 👶

making data 001

prereq to making data the datamade way for new hires/etl beginners

to-do

  • the essential bash
  • python patterns for custom transform code
  • tips of the iceberg for: postgresql, sql the language™, geospatial magic
@hancush
hancush / README.md
Last active March 16, 2017 15:59
pick yr bracket w wildly misinterpreted statistics scraped from espn.com

🏀🗑 march sadness

if u love ur friends but hate basketball, let the computer do your brack for you.

75 percent of the time it returns the better team as measured by power index ranking. 25 percent of the time it returns the underdog. for teams within 15 spots of one another, it returns whichever team has a color closest to my fave (bright yellow).

use

make sure to pip install colormath and also use python 3.x bc we're usin the input method!

then,

Index Crime CODE CATEGORY DESCRIPTION ILCS REFERENCE Index Crime Category Miscellaneous h i j
X 0110 Homicide First Degree Murder 720-5/9-1 Criminal homicide
0115 Homicide Intentional Homicide of an Unborn Child 720-5/9-1.2
0120 Homicide Voluntary Manslaughter of an Unborn Child 720-5/9-2.1
X 0130 Homicide Second Degree Murder 720-5/9-2 Criminal homicide
0141 Homicide Involuntary Manslaughter 720-5/9-3
0142 Homicide Reckless Homicide 720-5/9-3
0150 Homicide Justifiable Homicide 720-5/7-1
0160 Homicide Concealment of Homicidal Death 720-5/9-3.4
0165 Homicide Involuntary Manslaughter & Reckless Homicide of an Unborn Child 720-5/9-3.2
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 32 columns, instead of 4. in line 4.
id,source,site_name,address,zip,phone,fax,program_name,length_of_day,idhs_provider_id,agency,neighborhood,funded_enrollment,program_option,number_per_site_ehs,number_per_site_hs,director,head_start_fund,eearly_head_start_fund,cc_fund,progmod,website,executive_director,center_director,ece_available_programs,naeyc_valid_until,naeyc_program_id,email_address,ounce_of_prevention_description,purple_binder_service_type,column,column2
1965,DFSS_AgencySiteLies_2012.csv,"Chicago Public Schools Barton, Clara",7650 S. Wolcott Ave ,60620,5353260,,,,,,,Chicago Public Schools,,,,,,,,,,,www.cps.edu,Paula Cottone,Mr. Frank Gettridge,,,,,,
2649,ECE Chicago Find a School scrape.csv, McClellan,3527 S. Wallace ,60609,5351732,,,,,,Bridgeport,,,,,,,,,,,,,HS-HD,,,,,,,
482,CPS_Early_Childhood_Portal_scrape.csv, Chicago Commons Association - Taylor Center for New Experiences,1633 N Hamlin Ave ,,2278551,,Early Head Start,EXTENDED DAY,,,,,,,,,,,,,,,,,,,,,,,
2813,ECE Chicago Find a School scrape.csv, Thresholds Mothers' Project I/T,1110
@hancush
hancush / Makefile
Last active November 8, 2017 12:41
Diffing UCR codes
SHELL := bash
OUTFILES=new_code.csv different_statute.csv retired_code.csv
.INTERMEDIATE : joined_codes.csv changed_codes.csv
all : $(OUTFILES)
clean :
rm $(OUTFILES)
@hancush
hancush / lowes.csv
Last active May 18, 2022 20:49
👷 lowe's locations scraped from http://lowes.know-where.com/lowes/cgi/index
store_name store_number address city state zip_code
LOWE'S OF ALABASTER, AL 2525 235 COLONIAL PROMENADE PARKWAY ALABASTER AL 35007
LOWE'S OF N. ANNISTON, AL 2659 4901 MCCLELLAN BOULEVARD ANNISTON AL 36206
LOWE'S OF ATHENS, AL 1799 1109 US HIGHWAY 72 EAST ATHENS AL 35611
LOWE'S OF BESSEMER, AL 1531 1201 19TH STREET NORTH BESSEMER AL 35020
LOWE'S OF TRUSSVILLE, AL 0594 1885 EDWARDS LAKE RD. BIRMINGHAM AL 35235
LOWE'S OF INVERNESS, AL 1726 5291 HIGHWAY 280 BIRMINGHAM AL 35242
LOWE'S OF CULLMAN, AL 0313 1717 CHEROKEE AVENUE, SW CULLMAN AL 35055
LOWE'S OF DAPHNE, AL 2251 29645 FREDERICK BOULEVARD DAPHNE AL 36526
LOWE'S OF DECATUR, AL 1176 1641 BELTLINE ROAD SW DECATUR AL 35601
16:42:54 INFO scrapelib: GET - https://webapi.legistar.com/v1/nyc/matters/53903/texts/55398
Traceback (most recent call last):
  File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 280, in recv_into
    return self.connection.recv_into(*args, **kwargs)
  File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/OpenSSL/SSL.py", line 1625, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/OpenSSL/SSL.py", line 1431, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError