Skip to content

Instantly share code, notes, and snippets.


Ed Summers edsu

View GitHub Profile
View pinboard-ask.txt
Hi there,
My name is Maciej, I run the bookmarking site Pinboard, and I’m writing to ask
for your help.
You joined the site back when there was a one-time signup fee. Back then,
charging for bookmarking online was unheard of, and the fee was more of an
anti-spam measure than a revenue model.
In 2015, I changed Pinboard over to a subscription site, where even “basic”
edsu / CC-MAIN-2021-04-hosts-sizes-top100.csv
Last active Feb 7, 2021
The top 100 hosts by WARC record sizes (bytes) in commoncrawl CC-MAIN-2021-04.
View CC-MAIN-2021-04-hosts-sizes-top100.csv
url_host_name length 35553494127 22829413806 19523224128 18400799789 17043193709 16945185389 15723224197 15241728762 15017634948
edsu / CC-MAIN-2021-04-host-counts-top100.csv
Last active Feb 7, 2021
The top 100 host counts in commoncrawl CC-MAIN-2021-04
View CC-MAIN-2021-04-host-counts-top100.csv
url_host_name total 1640422 1056353 543797 472899 433041 394818 338835 296540 290895
View gist:50a9ac542b2ade5a60dd833c9c31f878
Susan A. Davis,
Martha Roby,
"F. James Sensenbrenner, Jr.",
Joseph P. Kennedy III,
Denny Heck,
Donna E. Shalala,
Steve Watkins,
Kendra S. Horn,
Ben McAdams,
View most.yaml
- id:
bioguide: B001288
lis: S370
thomas: '02194'
govtrack: 412598
opensecrets: N00035267
votesmart: 76151
wikipedia: Cory Booker
ballotpedia: Cory Booker
View outgoing.csv
name twitter
Susan A. Davis
Martha Roby
F. James Sensenbrenner, Jr.
Joseph P. Kennedy III
Denny Heck
Donna E. Shalala
Steve Watkins
Kendra S. Horn
Ben McAdams
View outgoing.csv
We can make this file beautiful and searchable if this error is corrected: It looks like row 8 should actually have 9 columns, instead of 6. in line 7.
Lamar Alexander,,True,,True,,,,True
Michael B. Enzi,,True,,True,,True,,True
Pat Roberts,,True,,True,,True,,
Tom Udall,,True,,True,,True,,True
Justin Amash,,,,,,True,,
Rob Bishop,,True,,True,,True,,
Wm. Lacy Clay,,,,,
View most.yaml
bioguide: P000598
thomas: '01910'
govtrack: 412308
opensecrets: N00029127
votesmart: 106220
- H8CO02137
cspan: 1031300
wikipedia: Jared Polis
edsu /
Last active Jan 31, 2021
use twitter's search client to retrieve all possible data for a tweet from the v2 api
View \
--query obama \
--results-per-call 100 \
--tweet-fields id,text,attachments,author_id,context_annotations,conversation_id,created_at,entities,geo,in_reply_to_user_id,lang,possibly_sensitive,public_metrics,referenced_tweets,reply_settings,source,withheld \
--user-fields id,name,username,created_at,description,entities,location,pinned_tweet_id,profile_image_url,protected,public_metrics,url,verified,withheld \
--media-fields media_key,type,duration_ms,height,preview_image_url,public_metrics,width \
--poll-fields id,options,duration_minutes,end_datetime,voting_status \
--place-fields full_name,id,contained_within,country,country_code,geo,name,place_type \
--expansions author_id,,in_reply_to_user_id,attachments.media_keys,attachments.poll_ids,geo.place_id,entities.mentions.username, \
--filename-prefix obama \