Skip to content

Instantly share code, notes, and snippets.

Ed Summers edsu

View GitHub Profile
View bills.py
import json
import requests
# get maryland's current house members
key = "your_key_here"
headers = {"X-API-KEY": key}
url = 'https://api.propublica.org/congress/v1/members/house/md/current.json'
resp = requests.get(url, headers=headers)
View get.py
import requests
key = "2B0i2oVrIov0sn69IL56GLcInvHqzv90E0U2kJLQ"
headers = {"X-API-KEY": key}
url = 'https://api.propublica.org/congress/v1/116/house/members.json'
resp = requests.get(url, headers=headers)
print(resp.json())
View get.py
import requests
key = "my_key"
headers = {"X-API-KEY": key}
url = 'https://api.propublica.org/congress/v1/116/house/members.json'
resp = requests.get(url, headers=headers)
print(resp.json())
View retweet_ratio.py
#!/usr/bin/env python3
import sys
from twarc import Twarc
screen_name = sys.argv[1]
twitter = Twarc()
tweets = 0
View obama-ids.txt
1113160564858122240
1113160564438650880
1113160563729862661
1113160562849054721
1113160560227627018
1113160559212421120
1113160556897292288
1113160556737957895
1113160555966197760
1113160555697668096
View blm.py
#!/usr/bin/env python3
import re
import csv
import json
import time
from requests_html import HTMLSession
def main():
@edsu
edsu / test.py
Last active Mar 27, 2019
Reading a string containing MARCXML with pymarc.
View test.py
from pymarc.marcxml import parse_xml_to_array
from io import StringIO
xml_text = open('test/batch.xml').read()
xml = StringIO(xml_text)
records = parse_xml_to_array(xml)
print(records)
View gist:475ab8e2f3307a5854faaf7fd06a1c02
[edsu@r001 ~]$ spark-shell --packages "io.archivesunleashed:aut:0.17.0"Ivy Default Cache set to: /home/edsu/.ivy2/cache
The jars for the packages stored in: /home/edsu/.ivy2/jars
:: loading settings :: url = jar:file:/home/edsu/.local/lib/python2.7/site-packages/pyspark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
io.archivesunleashed#aut added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-2ae7a372-bba6-482d-8fff-910ec07724ab;1.0
confs: [default]
found io.archivesunleashed#aut;0.17.0 in local-m2-cache
:: resolution report :: resolve 234ms :: artifacts dl 13ms
:: modules in use:
io.archivesunleashed#aut;0.17.0 from local-m2-cache in [default]
View russian-tweets.csv
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 7.
user,tweet text
"cellsstitr","RT realJamesAllsup: I was in the thick of it. I did not see any #UniteTheRight guys start violence. Violence was started by leftists & Anti…"
"beeatrwl","RT RapinBill: Liberals are triggered by a #UniteTheRight Pre-Rally, but totally fine with Black Lives Matter gunning down cops... #Charlot…"
"marrissatrr","#mar RT RodStryker: TheJusticeDept #UniteTheRight DOESN'T represent me, nor MILLIONS of other Trump supporters. We don't condone racia… …"
"marrissatrr","#mar RT RodStryker: Sessions TheJusticeDept Fed Investigation #Charlottesville >
EXPOSE Soros funded #Antifa #BLM & opp #UniteTheRight… …"
"mayluusstr","#topl RT blazebandit2015: Marine #Veteran Reveals the #TRUTH about the #Charlottesville Tragedy.
#Antifa #Soros #TrumpArmy #UniteTheRight …"
"elizeestr","RT NamesNotBecky: The horror straight from the #UniteTheRight rally
@edsu
edsu / russian-accounts-unitetheright.csv
Last active Mar 10, 2019
Number of tweets in a 200,113 tweet #unitetheright dataset (collected August 15, 2017) sent by users identified as Russian trolls in https://intelligence.house.gov/uploadedfiles/exhibit_b.pdf
View russian-accounts-unitetheright.csv
screen_name tweets
elizeestr 6
mayluusstr 5
TEN_GOP 5
sterrsam 4
marrissatrr 4
thelmmisb 2
elinsstr 2
aswwimMOrris 2
verosanrrt 1
You can’t perform that action at this time.