Skip to content

Instantly share code, notes, and snippets.

@mccutchen

mccutchen/thresholderbot_twitter_usage_analysis.md

Last active July 19, 2023 19:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mccutchen/3416442743b8ae20a502ae367dba43f9 to your computer and use it in GitHub Desktop.
Save mccutchen/3416442743b8ae20a502ae367dba43f9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
thresholderbot_twitter_usage_analysis.md

Quick and dirty Twitter API pricing analysis for Thresholderbot

Summary

Twitter's new API pricing starts at $5,000 per month to read 1 million tweets per month. According to our metrics, Thresholderbot read more than 15 million tweets per month on average over the last 12 full months of activity — and that estimate may be far too low if retweets are counted as 2 reads.

Tweets read by Thresholderbot every 30 days over the past year

graph image

Details

Our implementation fetches each Thresholderbot user's "reverse chronological" timeline every ~90 seconds. That's one Twitter API request per user per 90 second interval1, but each one of those API requests may return 0 or more individual tweets (depending on how many accounts each user follows, how frequently those accounts tweet, etc).

According to the Twitter API "Tweet Caps" documentation, if a single API request returns 100 tweets from a user's timeline, that would count as 100 reads against the allowance. For bonus points, every retweet returned in a user's timeline counts as two reads according to this forum post.

Given that 2x penalty for retweets2, the 15M tweets per month reported above is actually much lower than what we'd be charged for if we paid for API access, because a significant proportion of tweets in the average user's timeline are retweets.

Our own monthly costs

For the record, Thresholderbot costs about $230/month to run, depending on the number of emails sent in each month. That cost breaks down like so:

Service Cost per month Notes
AWS (Amazon Web Services) $200 Infrastructure provider (EC2, RDS, S3)
Mailgun $12-20 Outbound email to users (35,000 – 45,000 per month)
Google $12 G Suite for @thresholderbot.com email accounts

Our AWS spend in particular could be optimized by simplifying Thresholderbot's hilariously over-engineered, overly complex codebase and architecture, but … that effort would be wasted now!

Emails sent to Thresholderbot users every 30 days over the past year

graph image

Footnotes

  1. Note: that 90s interval was specifically chosen to stay well under the rate limits for that API endpoint to make sure we're not abusing Twitter's systems! ↩

  2. On a technical level, I understand why they would charge 2x views for each retweet, since each retweet is literally returned as two tweets in a trenchcoat. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment