Skip to content

Instantly share code, notes, and snippets.

View jackbandy's full-sized avatar
🎓

Jack Bandy jackbandy

🎓
View GitHub Profile
phrase total_occurrences bias_score p_dem p_rep n_dem n_rep n_senators
il today 171 -0.9844844254757246 0.030395136778115502 0.0002376425855513308 170 1 7
fossil fuel 223 -0.9762291824717316 0.03951367781155015 0.0004752851711026616 221 2 21
climate crisis 108 -0.975461172449649 0.019131056677990345 0.0002376425855513308 107 1 22
aca 193 -0.9725467963490627 0.03414983014482389 0.0004752851711026616 191 2 83
student debt 79 -0.9664906221073223 0.013946003933488289 0.0002376425855513308 78 1 16
health disparities 75 -0.9297295599459747 0.013052029322367245 0.0004752851711026616 73 2 23
ma 665 -0.9208356628757051 0.1153227248346147 0.004752851711026616 645 20 99
il 829 -0.9143400966638466 0.1433935276238155 0.006416349809885932 802 27 99
affordable care 271 -0.9123525954946132 0.0466654747005185 0.0021387832699619773 261 9 9
phrase total_occurrences bias_score p_dem p_rep n_dem n_rep n_senators
nd issued 57 0.9734858986314489 0.00017879492222420883 0.013307984790874524 1 56 17
unborn 207 0.97078312913173 0.0007151796888968353 0.048241444866920155 4 203 30
energy producers 80 0.9430434145075968 0.0005363847666726265 0.018298479087452472 3 77 11
largest economic 52 0.9415689460054293 0.00035758984444841767 0.01188212927756654 2 50 6
enzi 147 0.9274907812661544 0.0012515644555694619 0.03326996197718631 7 140 4
neb 84 0.8892410546841583 0.001072769533345253 0.018298479087452472 6 77 4
cattle producers 80 0.8850089950584108 0.001072769533345253 0.01758555133079848 6 74 18
liberal 70 0.845705380878174 0.0012515644555694619 0.01497148288973384 7 63 25
communist 818 0.8354249709933904 0.015555158233506169 0.17347908745247148 87 730 21
phrase total_occurrences
health 21070
senator 17989
president 16663
act 15494
senate 13020
19 13017
covid 12863
covid 19 12513
said 12391
username
0 judgejeanine
1 jim_jordan
2 mariabartiromo
3 vp
4 gopchairwoman
5 parscale
6 presssec
7 tuckercarlson
8 jessebwatters
url date source
https://apnews.com/bdc1c8033fee26acd9d02153c0687121 2020-03-13T21:43:38Z Associated Press
https://www.techradar.com/news/how-to-watch-emma-stream-the-2020-movie-online-anywhere 2020-03-25T11:30:26Z TechRadar
https://thehill.com/homenews/campaign/494237-biden-leads-trump-in-michigan-poll 2020-04-23T00:19:24Z The Hill
https://www.cnbc.com/2020/04/08/as-trump-attacks-who-warns-against-politicizing-coronavirus-if-you-dont-want-many-more-body-bags.html 2020-04-08T16:11:00Z CNBC
https://mspoweruser.com/kojima-productions-gdc-cancellation-coronavirus/ 2020-02-24T22:19:00Z Mspoweruser.com
https://www.cnbc.com/2020/03/23/these-banks-are-offering-coronavirus-financial-aid.html 2020-03-23T16:11:25Z CNBC
https://www.reuters.com/article/us-tennis-tennis-nadal-djokovic-federer-idUSKBN21F0TP 2020-03-28T17:36:39Z Reuters
https://www.vice.com/en_us/article/pke4py/best-songs-march-2020 2020-03-27T16:44:32Z ViceNews
... ... ...
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 7.
short_title,full_title,url
Biden leads Trump in Michigan: poll,Biden leads Trump in Michigan: poll | TheHill,https://thehill.com/homenews/campaign/494237-biden-leads-trump-in-michigan-poll
Tennis stars rally in fight against coronavirus,Tennis stars rally in fight against coronavirus - Reuters,https://www.reuters.com/article/us-tennis-tennis-nadal-djokovic-federer-idUSKBN21F0TP
Kojima Productions cancels GDC appearance over coronavirus worries,Kojima Productions cancels GDC appearance over coronavirus worries - MSPoweruser,https://mspoweruser.com/kojima-productions-gdc-cancellation-coronavirus/
Apple Has a Bright Future,Apple Has a Bright Future,https://gizmodo.com/apple-has-a-bright-future-1843187742
How restaurants are adapting to uncertainty,Coronavirus: How restaurants are adapting to uncertainty,https://www.usatoday.com/story/money/2020/03/17/coronavirus-how-restaurants-adapting-uncertainty/5057617002/
"Free food, delivery, deals from Chipotle and more","Burrito Day 2020: Free food, delivery, deals from
'''
headline_scraper.py
A simple scrapy spider to collect web page titles
'''
import scrapy
from pandas import read_csv
from readability.readability import Document
PATH_TO_DATA = 'https://gist.githubusercontent.com/jackbandy/208028b404d8c6a6f822397e306a5a34/raw/ef7f73357e77c29c63b5b7632d840a923327e179/100_urls_sample.csv'
We can make this file beautiful and searchable if this error is corrected: It looks like row 10 should actually have 3 columns, instead of 1. in line 9.
url,date,source
https://apnews.com/bdc1c8033fee26acd9d02153c0687121,2020-03-13T21:43:38Z,Associated Press
https://www.techradar.com/news/how-to-watch-emma-stream-the-2020-movie-online-anywhere,2020-03-25T11:30:26Z,TechRadar
https://thehill.com/homenews/campaign/494237-biden-leads-trump-in-michigan-poll,2020-04-23T00:19:24Z,The Hill
https://www.cnbc.com/2020/04/08/as-trump-attacks-who-warns-against-politicizing-coronavirus-if-you-dont-want-many-more-body-bags.html,2020-04-08T16:11:00Z,CNBC
https://mspoweruser.com/kojima-productions-gdc-cancellation-coronavirus/,2020-02-24T22:19:00Z,Mspoweruser.com
https://www.cnbc.com/2020/03/23/these-banks-are-offering-coronavirus-financial-aid.html,2020-03-23T16:11:25Z,CNBC
https://www.reuters.com/article/us-tennis-tennis-nadal-djokovic-federer-idUSKBN21F0TP,2020-03-28T17:36:39Z,Reuters
https://www.vice.com/en_us/article/pke4py/best-songs-march-2020,2020-03-27T16:44:32Z,ViceNews
https://www.cnn.com/videos/travel/2020/04/23/uk-london-covid-19-coronavirus-pandemic-tourism-guid
@jackbandy
jackbandy / analyze-2020-senate.py
Created March 21, 2020 20:45
Based on Renzo Lucioni's famous senate voting graphs https://gist.github.com/rlucioni/8bdb1092579041ce739c
import re
import pandas as pd
import networkx as nx
import urllib.request
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import matplotlib.animation as animation
from textwrap import wrapk
---
title: "omnidroid-bias"
author: "Jack Bandy"
date: "2/29/2020"
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```