Skip to content

Instantly share code, notes, and snippets.

Ed Summers edsu

Block or report user

Report or block edsu

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@edsu
edsu / replies.py
Last active Sep 16, 2019
Try to get replies to a particular set of tweets, recursively.
View replies.py
#!/usr/bin/env python
"""
Twitter's API doesn't allow you to get replies to a particular tweet. Strange
but true. But you can use Twitter's Search API to search for tweets that are
directed at a particular user, and then search through the results to see if
any are replies to a given tweet. You probably are also interested in the
replies to any replies as well, so the process is recursive. The big caveat
here is that the search API only returns results for the last 7 days. So
View toni-morrison-tweeted-urls.csv
We can't make this file beautiful and searchable because it's too large.
url,tweets
https://www.nytimes.com/2019/08/06/books/toni-morrison-dead.html,5099
https://www.vulture.com/2019/08/toni-morrison-dead-at-88.html,4222
https://www.theguardian.com/books/2019/aug/06/toni-morrison-author-and-pulitzer-winner-dies-aged-88,2118
https://www.npr.org/2019/08/06/542391535/toni-morrison-whose-soaring-novels-were-rooted-in-black-lives-dies-at-88?t=1565529065738,2009
https://www.washingtonpost.com/local/obituaries/toni-morrison-nobel-laureate-who-transfigured-american-literature-dies-at-88/2019/08/06/49cd4d46-b84d-11e9-a091-6a96e67d9cce_story.html,1463
https://www.marianne.net/politique/muriel-penicaud-reussit-l-hommage-toni-morrison-le-plus-insultant-de-l-histoire,1186
https://www.newyorker.com/magazine/2003/10/27/ghosts-in-the-house,1176
https://elpais.com/cultura/2019/08/06/actualidad/1565098762_496430.html,1171
https://www.newyorker.com/magazine/2016/11/21/making-america-white-again,1159
View http-response.txt
% curl -i https://twitter.com/BernieSanders/status/1161005398817722368
HTTP/2 302
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
content-length: 122
content-type: text/html;charset=utf-8
date: Tue, 13 Aug 2019 11:10:07 GMT
expires: Tue, 31 Mar 1981 05:00:00 GMT
last-modified: Tue, 13 Aug 2019 11:10:07 GMT
location: https://twitter.com/shaunking/status/1161003857176084480
@edsu
edsu / feral-hogs-urls.csv
Last active Aug 6, 2019
% twarc search "feral hogs" | unshrtn.py | urls.py | grep -v twitter.com | sort | uniq -c | sort -rn | csv > feral-hogs-urls.csv
View feral-hogs-urls.csv
tweet_count url
304 https://www.gq.com/story/hogblog
32 https://www.wideopenspaces.com/texas-losing-war-feral-hogs-2/
26 https://globalnews.ca/news/2630688/radioactive-wild-boars-overrun-northern-japan-5-years-after-nuclear-disaster/
24 https://www.youtube.com/watch?v=DJFFls6mBb8
19 https://www.thecut.com/2019/08/what-to-name-your-30-to-50-feral-hogs.html
19 https://knowyourmeme.com/memes/feral-hogs
14 https://splinternews.com/hog-blog-1836979161
10 https://www.outsideonline.com/2141891/attack-radioactive-wild-boars
8 https://wvupressonline.com/node/780
View jupyterhub_config.py
# Configuration file for jupyterhub.
import os
#------------------------------------------------------------------------------
# Configurable configuration
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
# LoggingConfigurable configuration
#------------------------------------------------------------------------------
View snowflakes.py
#!/usr/bin/env python3
import twarc
# This small script shows how to listen to the Twitter sample stream and
# deconstruct tweet ids into their various components. The tweet_components
# method accepts a tweet id and returns a dict object with key / values
# representing the various components of a tweet id. Each component has its own
# method detailing how values are extracted from the tweet id.
View langdetect_test.py
#!/usr/bin/env python
import csv
from xml.etree import ElementTree
from langdetect import detect_langs
from requests_html import HTMLSession
http = HTMLSession()
def langs(url):
@edsu
edsu / modern-titles-alpha.txt
Created Jun 6, 2019
csvcut -c 2 NSRLProd.txt | perl -nle 's/"//g; print uc(substr($_, 0, 1))' | sort | uniq -c > modern-titles-alpha.txt
View modern-titles-alpha.txt
2 <
60
14 .
6 {
2 ?
3 0
118 1
27 2
50 3
43 4
@edsu
edsu / ios-titles-alpha.txt
Last active Jun 6, 2019
csvcut -c 2 NSRLProd.txt | perl -nle 's/"//g; print uc(substr($_, 0, 1))' | sort | uniq -c > ios-titles-alpha.txt
View ios-titles-alpha.txt
3 `
1 .
4 1
1 6
1 8
88 A
17 B
35 C
22 D
9 E
View followers.py
#!/usr/bin/env python
import sys
import twarc
# usage: followers.py <screen_name>
screen_name = sys.argv[1]
twitter = twarc.Twarc()
You can’t perform that action at this time.