Skip to content

Instantly share code, notes, and snippets.

Ed Summers edsu

Block or report user

Report or block edsu

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View timeline-history.csv
user quotes retweets tweets total
doctorow 86 766 1678 2530
TalibKweli 910 286 466 1662
archillect 0 86 1114 1200
carlfranzen 82 872 30 984
nevali 177 334 264 775
DocDre 35 538 186 759
generativist 81 256 329 666
deray 20 585 61 666
tressiemcphd 211 211 213 635
View bucket-prefix-sizes.py
#!/usr/bin/env python3
import re
import boto3
from collections import Counter
from humanize import naturalsize
s3_client = boto3.client('s3')
View chatty-20200513090723-20200513174235.csv
user retweets tweets total
doctorow 39 130 169
carlfranzen 113 11 124
publicknowledge 74 40 114
byDVNLLN 48 36 84
socialistdogmom 2 76 78
Hungryghoast 60 18 78
blkwomenradical 54 15 69
dwallacewells 27 39 66
shaunking 60 5 65
View links.py
@edsu
edsu / tags.csv
Last active Apr 27, 2020
twarc search.log search 'endtheshutdown OR endlockdown OR endthelockdown OR operationgridlock' | utils/tags.py > tags.csv
View tags.csv
hashtag tweets
endthelockdown 13569
operationgridlock 4982
endtheshutdown 4736
coronavirus 3322
covid19 3256
lockdown 1621
endlockdown 1579
neilferguson 1390
endthelockdownnow 1306
View response.txt
curl -i https://mith.umd.edu/minimaldigipub/
HTTP/1.1 301 Moved Permanently
Server: GitHub.com
Content-Type: text/html
Location: https://umd-mith.github.io/minimaldigipub/
X-GitHub-Request-Id: D73A:34A8:110BE9:158E5F:5E873E74
Content-Length: 162
Date: Fri, 03 Apr 2020 13:48:33 GMT
Via: 1.1 varnish
View propublica-reddit.csv
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 8 columns, instead of 7. in line 4.
id,created,creator,score,comments,url,title,fld
fjuuho,2020-03-16 19:33:32,Canuknucklehead,53,118,https://www.propublica.org/article/no-matter-what-some-public-officials-say-the-message-you-need-to-hear-is-stay-home,"No Matter What Some Public Officials Say, the Message You Need to Hear Is “Stay Home”",propublica.org
fpezvu,2020-03-26 12:54:41,OldFashionedJizz,45,24,https://www.propublica.org/article/internal-emails-show-how-chaos-at-the-cdc-slowed-the-early-response-to-coronavirus,Internal Emails Show How Chaos at the CDC Slowed the Early Response to Coronavirus,propublica.org
flk03a,2020-03-19 19:14:10,mybadselves,20,24,https://www.propublica.org/article/senator-dumped-up-to-1-6-million-of-stock-after-reassuring-public-about-coronavirus-preparedness,Senator Dumped Millions in Stock After Coronavirus Briefings - Rolling Stone,propublica.org
fpjucr,2020-03-26 17:15:36,numberphifteen,16,2,https://www.propublica.org/article/how-china-built-a-twitter-propaganda-machine-then-let-it-loose-on-coronavirus,How China
View propublica-reddit.csv
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 8 columns, instead of 5. in line 4.
id,created,creator,score,comments,fld,url,title
fpufdb,https://www.propublica.org/article/people-with-intellectual-disabilities-may-be-denied-lifesaving-care-under-these-plans-as-coronavirus-spreads,People With Intellectual Disabilities May Be Denied Lifesaving Care Under These Plans as Coronavirus Spreads,2020-03-27 06:09:36,iyoiiiu,1,95,propublica.org
fplzes,https://www.propublica.org/article/our-goal-should-be-to-crush-the-curve,"""Our Goal Should Be to Crush the Curve"" - ProPublica",2020-03-26 19:19:40,dect60,1,6,propublica.org
fpjucr,https://www.propublica.org/article/how-china-built-a-twitter-propaganda-machine-then-let-it-loose-on-coronavirus,How China Built a Twitter Propaganda Machine Then Let It Loose on Coronavirus,2020-03-26 17:15:36,numberphifteen,16,2,propublica.org
fpezvu,https://www.propublica.org/article/internal-emails-show-how-chaos-at-the-cdc-slowed-the-early-response-to-coronavirus,Internal Emails Show How Chaos at the CDC Slowed the Early Response to Coronavirus,2020-03-26 12:54:41,OldF
View time_test.py
#!/usr/bin/env python3
"""
Twitter's rate limits allow App Auth contexts to search at 450 requests
every 15 minutes, and User Auth contexts at 180 requests per 15 minutes.
This script exercises both contexts and counts how tweets it is able to
receive. We should see a significant number more tweets coming back for App
Auth.
Typical output should look like:
View archiveit-covid19.py
#!/usr/bin/env python3
import csv
import requests
url = 'https://partner.archive-it.org/api/seed'
params = {
"collection": 13529,
"limit": 100,
"offset": 0
You can’t perform that action at this time.