Skip to content

Instantly share code, notes, and snippets.

Ed Summers edsu

View GitHub Profile
@edsu
edsu / replies.py
Last active Jul 30, 2020
Try to get replies to a particular set of tweets, recursively.
View replies.py
#!/usr/bin/env python
"""
Twitter's API doesn't allow you to get replies to a particular tweet. Strange
but true. But you can use Twitter's Search API to search for tweets that are
directed at a particular user, and then search through the results to see if
any are replies to a given tweet. You probably are also interested in the
replies to any replies as well, so the process is recursive. The big caveat
here is that the search API only returns results for the last 7 days. So
@edsu
edsu / email.txt
Created Jun 30, 2020
received at info@docnow.io
View email.txt
How have you been?
I hope you, your family, and friends stay safe by taking the proper precuations
as we enter in to confusing and troubling times as we watch a worldwide
pandemic unfold with this Coronavirus.
Could there be something more deeper going on with all of this? Could we be
seeing the signs of the times unfolding? Could this be a door of opportunity
for people to open their hearts to what God has to say?
@edsu
edsu / bots.py
Created Jun 23, 2011
see what your bot traffic is like
View bots.py
#!/usr/bin/env python
"""
Hack to look for user agent strings in typical Apache style log and
count up the number of requests by bots vs non-bots. The list of bot
user agents comes from http://www.user-agents.org/ but has had some
agents added to it, since the user-agents.org list is pretty out of date, e.g.
no Bing?!
"""
View payload.json
{
"globalObjects": {
"broadcasts": {},
"cards": {},
"lists": {},
"media": {},
"moments": {},
"places": {},
"topics": {},
"tweets": {
@edsu
edsu / top10-comparison.csv
Last active May 28, 2020
Comparing the top co-occurring tweets in two sets of #GeorgeFloyd tweets. A set of 100,000 tweets and a set of 1000 tweets.
View top10-comparison.csv
100k 1k
blacklivesmatter blacklivesmatter
justiceforgeorgefloyd justiceforgeorgefloyd
minneapolis minneapolis
minneapolisriots minneapolisriots
icantbreathe icantbreathe
blacklivesmatters georgefloydwasmurdered
georgefloydwasmurdered breonnataylor
racism ahmaudarbery
minneapolispd blacklivesmatters
@edsu
edsu / georgefloyd-hashtags.csv
Last active May 28, 2020
Hashtags counts in the last 116,901 tweets that use the hashtag #GeorgeFloyd.
View georgefloyd-hashtags.csv
hashtag count
georgefloyd 116901
blacklivesmatter 28047
justiceforgeorgefloyd 11167
minneapolis 9190
minneapolisriots 6193
icantbreathe 5380
blacklivesmatters 2792
georgefloydwasmurdered 2667
racism 2387
@edsu
edsu / user_mentions.py
Last active May 27, 2020
Collect tweet JSON for tweets that mention particular users.
View user_mentions.py
#!/usr/bin/env python3
"""
Listen for tweets mentioning particular users.
"""
import json
import twarc
twitter = twarc.Twarc()
View timeline-history.csv
user quotes retweets tweets total
doctorow 86 766 1678 2530
TalibKweli 910 286 466 1662
archillect 0 86 1114 1200
carlfranzen 82 872 30 984
nevali 177 334 264 775
DocDre 35 538 186 759
generativist 81 256 329 666
deray 20 585 61 666
tressiemcphd 211 211 213 635
@edsu
edsu / wpd2txt.py
Last active May 20, 2020
convert a wordperfect file to messy text
View wpd2txt.py
#!/usr/bin/env python
import sys
"""
For each character c, if its value is:
#13: treat as newline
#128, #160: treat as space ' '
#169..#171, #173, #174: treat as dash '-'
#192..#236: skip ahead and ignore all characters until another occurrence
View bucket-prefix-sizes.py
#!/usr/bin/env python3
import re
import boto3
from collections import Counter
from humanize import naturalsize
s3_client = boto3.client('s3')
You can’t perform that action at this time.