Skip to content

Instantly share code, notes, and snippets.

View jheasly's full-sized avatar

John Heasly jheasly

View GitHub Profile

Keybase proof

I hereby claim:

  • I am jheasly on github.
  • I am jpheasly (https://keybase.io/jpheasly) on keybase.
  • I have a public key ASBH6OHTrcv0xB0ZSBahB0MZCu7efJjvnDo4PKbkEGeWiQo

To claim this, I am signing this object:

@jheasly
jheasly / output.txt
Created March 16, 2021 21:02
Scrapy Oregon health inspection output
(open-health-inspection-scraper) bash-3.2$ ./scrapeHealthData.py
/Users/jpheasly/Development/open-health-inspection-scraper/scraper/spiders/healthspace_spider.py:206: SyntaxWarning: "is" with a literal. Did you mean "=="?
'critical': critical is "critical",
2021-03-16 15:49:34 [scrapy.utils.log] INFO: Scrapy 1.3.3 started (bot: scrapybot)
2021-03-16 15:49:34 [scrapy.utils.log] INFO: Overridden settings: {'DOWNLOAD_DELAY': 10, 'SPIDER_MODULES': ['scraper.spiders']}
2021-03-16 15:49:34 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.spiderstate.SpiderState']
mydate,cday,dday,ma
2020-03-01,0.0,0.0,
2020-03-02,0.0,0.0,
2020-03-03,0.0,0.0,
2020-03-04,0.0,0.0,
2020-03-05,0.0,0.0,
2020-03-06,0.0,0.0,
2020-03-07,0.0,0.0,0.0
2020-03-08,1.0,0.0,0.14285714285714285
2020-03-09,0.0,0.0,0.14285714285714285
mydate,cday,dday,ma
2020-03-01,0.0,0.0,
2020-03-02,0.0,0.0,
2020-03-03,0.0,0.0,
2020-03-04,0.0,0.0,
2020-03-05,0.0,0.0,
2020-03-06,0.0,0.0,
2020-03-07,0.0,0.0,0.0
2020-03-08,1.0,0.0,0.14285714285714285
2020-03-09,0.0,0.0,0.14285714285714285
mydate cday dday ma
2020-03-01 0.0 0.0
2020-03-02 0.0 0.0
2020-03-03 0.0 0.0
2020-03-04 0.0 0.0
2020-03-05 0.0 0.0
2020-03-06 0.0 0.0
2020-03-07 0.0 0.0 0.0
2020-03-08 0.0 0.0 0.0
2020-03-09 0.0 0.0 0.0
@jheasly
jheasly / get_the_API.py
Last active March 9, 2021 01:01
An example of from using Python to get the API behind a Tableau site, rather than scraping the HTML. Per the author, Jeremy J. Bowers, it will work for any Tableau vizql implementation, with a bit of URL substitution.
# Related, from News Nerdery thread:
#
# aricchokey 2 hours ago
# @ejmurra Looks like another way to grab the data is by tacking on a ".csv" at the end of the chosen sheet. It will
# return the delimited version of the data/trigger the download from Tableau, too.
# Like https://bi.ahca.myflorida.com/t/ABICC/views/Public/HospitalBedsHospital.csv or
# https://bi.ahca.myflorida.com/t/ABICC/views/Public/ICUBedsCounty.csv. Might get rid of the need for a payload
# in your script.
from bs4 import BeautifulSoup
@jheasly
jheasly / tarbell_serve_error.py
Created March 6, 2019 04:48
Tarbell traceback
Traceback (most recent call last):
File "/Users/jpheasly/.virtualenvs/tarbell/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/jpheasly/.virtualenvs/tarbell/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/jpheasly/.virtualenvs/tarbell/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/jpheasly/.virtualenvs/tarbell/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Users/jpheasly/.virtualenvs/tarbell/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
@jheasly
jheasly / docker_exec_mongo.md
Last active November 9, 2017 01:48
The warnings from the mongo inside the Talk Docker container
bash-3.2$ docker exec -it [container ID] mongo
MongoDB shell version v3.4.10
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.10
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
	http://docs.mongodb.org/
Questions? Try the support group
@jheasly
jheasly / django_scheduler_event_generic_relation_example.py
Last active September 5, 2016 20:03
civic_calendar & django-scheduler
'''
Experiments in extending the django_scheduler Event class via the built-in EventRelation class.
'''
>>> from schedule.models import Calendar, Event, EventRelation
>>> from civic_calendar.models import Meeting
>>> event = Event.objects.get(pk=2)
>>> meeting = Meeting.objects.get(pk=2)
>>> er = EventRelation(event=event, content_object=meeting)
>>> er.save()
@jheasly
jheasly / bat_avg.py
Last active April 27, 2016 01:46
mlbgame gets batting avg.
>>> from __future__ import print_function
>>> import mlbgame
>>> game = mlbgame(2016, 4, 26, home='Mariners'][0]
>>> stats = mlbgame.player_stats(game.game_id)
>>> for player in stats['home_batting']:
... print(player.name_display_first_last, player.avg)
...
Norichika Aoki 0.205
Ketel Marte 0.238
Robinson Cano 0.224