Skip to content

Instantly share code, notes, and snippets.

@drench
Last active February 26, 2018 03:01
Show Gist options
  • Save drench/21365802b5226bb6eb9b9a0975b06fcf to your computer and use it in GitHub Desktop.
Save drench/21365802b5226bb6eb9b9a0975b06fcf to your computer and use it in GitHub Desktop.
Fetch your twitter timeline, one JSON blob per file
#!/bin/sh
# https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-home_timeline
since_id() {
if [ -r './.since_id' ]; then
cat './.since_id'
else
echo 100000
fi
}
fetch_timeline() {
raw_out=$(mktemp)
url="/1.1/statuses/home_timeline.json?count=200&since_id=$(since_id)"
echo "calling $url and sending output to $raw_out"
twurl $url > $raw_out
jq -r 'sort_by(.id) | .[].id' < $raw_out | while read id; do
echo $id > './.since_id'
tweetfile="tweet-${id}.json"
if [ -e "$tweetfile" ]; then
echo "Skipping already-stored tweet $id"
else
echo "Storing new tweet $id"
jq ".[] | select(.id == ${id})" < $raw_out > $tweetfile
fi
done
}
fetch_timeline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment