Skip to content

Instantly share code, notes, and snippets.

View hancush's full-sized avatar
🍓

hannah cushman garland hancush

🍓
View GitHub Profile
from django.db.models import Q
condition = Q()
employers = Employer.objects.all()
if params:
if params.get('name'):
employers = employers.annotate(search_vector=SearchVector('name'))
TABLES=$(mdb-tables /path/to/YOUR_ACCDB.accdb); \
for TABLE in $TABLES; \
do mdb-export /path/to/YOUR_ACCDB.accdb $TABLE > /path/to/$TABLE.csv; \
done;
@hancush
hancush / django-store-files-s3.md
Last active March 7, 2018 15:53
store django model-related files in s3

Install django-storages:

pip install django-storages

Add to settings:

INSTALLED_APPS = (
# header
here's what this is
<<>>=
import pandas as pd
@
<<>>=
data = pd.read_csv('path/to/your/data') # returns a dataframe

Where test.txt contains:

a string      with some  extra spaces




line
from make_a_formset import formset
# populate your formset
for form in formset:
form_data = form.clean_data
# {'name': 'Hannah', 'favorite_color': 'yellow'}
# do stuff with the data
  1. Assemble web_info by iterating over table cells parsed from Legistar GUI

  2. Assemble terms by iterating over City Council terms from Legistar API, adding members to the web_info dict as we go

  3. Assemble members by iterating over terms, storing names as keys, and Person objects, loaded with memberships for each City Council term, as values

  4. Continue to assemble members by iterating over body offices for each committee, storing memberships on the appropriate Person objects (or minting

16:42:54 INFO scrapelib: GET - https://webapi.legistar.com/v1/nyc/matters/53903/texts/55398
Traceback (most recent call last):
  File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 280, in recv_into
    return self.connection.recv_into(*args, **kwargs)
  File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/OpenSSL/SSL.py", line 1625, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/OpenSSL/SSL.py", line 1431, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError
@hancush
hancush / lowes.csv
Last active May 18, 2022 20:49
👷 lowe's locations scraped from http://lowes.know-where.com/lowes/cgi/index
store_name store_number address city state zip_code
LOWE'S OF ALABASTER, AL 2525 235 COLONIAL PROMENADE PARKWAY ALABASTER AL 35007
LOWE'S OF N. ANNISTON, AL 2659 4901 MCCLELLAN BOULEVARD ANNISTON AL 36206
LOWE'S OF ATHENS, AL 1799 1109 US HIGHWAY 72 EAST ATHENS AL 35611
LOWE'S OF BESSEMER, AL 1531 1201 19TH STREET NORTH BESSEMER AL 35020
LOWE'S OF TRUSSVILLE, AL 0594 1885 EDWARDS LAKE RD. BIRMINGHAM AL 35235
LOWE'S OF INVERNESS, AL 1726 5291 HIGHWAY 280 BIRMINGHAM AL 35242
LOWE'S OF CULLMAN, AL 0313 1717 CHEROKEE AVENUE, SW CULLMAN AL 35055
LOWE'S OF DAPHNE, AL 2251 29645 FREDERICK BOULEVARD DAPHNE AL 36526
LOWE'S OF DECATUR, AL 1176 1641 BELTLINE ROAD SW DECATUR AL 35601