Install django-storages
:
pip install django-storages
Add to settings:
INSTALLED_APPS = (
from django.db.models import Q | |
condition = Q() | |
employers = Employer.objects.all() | |
if params: | |
if params.get('name'): | |
employers = employers.annotate(search_vector=SearchVector('name')) |
TABLES=$(mdb-tables /path/to/YOUR_ACCDB.accdb); \ | |
for TABLE in $TABLES; \ | |
do mdb-export /path/to/YOUR_ACCDB.accdb $TABLE > /path/to/$TABLE.csv; \ | |
done; |
Install django-storages
:
pip install django-storages
Add to settings:
INSTALLED_APPS = (
# header | |
here's what this is | |
<<>>= | |
import pandas as pd | |
@ | |
<<>>= | |
data = pd.read_csv('path/to/your/data') # returns a dataframe |
Where test.txt contains:
a string with some extra spaces
line
from make_a_formset import formset | |
# populate your formset | |
for form in formset: | |
form_data = form.clean_data | |
# {'name': 'Hannah', 'favorite_color': 'yellow'} | |
# do stuff with the data |
Assemble web_info
by iterating over table cells parsed from Legistar GUI
Assemble terms by iterating over City Council terms from Legistar API,
adding members to the web_info
dict as we go
Assemble members
by iterating over terms
, storing names as keys, and
Person objects, loaded with memberships for each City Council term, as values
Continue to assemble members
by iterating over body offices for each
committee, storing memberships on the appropriate Person objects (or minting
16:42:54 INFO scrapelib: GET - https://webapi.legistar.com/v1/nyc/matters/53903/texts/55398
Traceback (most recent call last):
File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 280, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/OpenSSL/SSL.py", line 1625, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/Users/Hannah/.virtualenvs/ocd/lib/python3.5/site-packages/OpenSSL/SSL.py", line 1431, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
store_name | store_number | address | city | state | zip_code | |
---|---|---|---|---|---|---|
LOWE'S OF ALABASTER, AL | 2525 | 235 COLONIAL PROMENADE PARKWAY | ALABASTER | AL | 35007 | |
LOWE'S OF N. ANNISTON, AL | 2659 | 4901 MCCLELLAN BOULEVARD | ANNISTON | AL | 36206 | |
LOWE'S OF ATHENS, AL | 1799 | 1109 US HIGHWAY 72 EAST | ATHENS | AL | 35611 | |
LOWE'S OF BESSEMER, AL | 1531 | 1201 19TH STREET NORTH | BESSEMER | AL | 35020 | |
LOWE'S OF TRUSSVILLE, AL | 0594 | 1885 EDWARDS LAKE RD. | BIRMINGHAM | AL | 35235 | |
LOWE'S OF INVERNESS, AL | 1726 | 5291 HIGHWAY 280 | BIRMINGHAM | AL | 35242 | |
LOWE'S OF CULLMAN, AL | 0313 | 1717 CHEROKEE AVENUE, SW | CULLMAN | AL | 35055 | |
LOWE'S OF DAPHNE, AL | 2251 | 29645 FREDERICK BOULEVARD | DAPHNE | AL | 36526 | |
LOWE'S OF DECATUR, AL | 1176 | 1641 BELTLINE ROAD SW | DECATUR | AL | 35601 |