Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Some recipes for the Twitter API for twurl +jq, and other useful Twitter code snippets and tools

twurl "/1.1/users/show.json?screen_name=andypiper" | jq '. | {id: .id_str, name: .name, screen_name: .screen_name, bio: .description, location: .location}'

{
  "id": "786491",
  "name": "andy piper (pipes)",
  "screen_name": "andypiper",
  "bio": "I'm the lead @TwitterDev @twitterapi platform team - find the best help via https://t.co/T2vkQoJ64f. Code, community, & respect. ⌨️ 🌈 🙇 #HeForShe 🏳️‍🌈",
  "location": "Kingston upon Thames, London"
}

twurl "/1.1/users/lookup.json?screen_name=andypiper,evilpiper,airatair" | jq '[.[] | {user_id: .id_str, screen_name: .screen_name}]'

[
  {
    "user_id": "786491",
    "screen_name": "andypiper"
  },
  {
    "user_id": "54931584",
    "screen_name": "evilpiper"
  },
  {
    "user_id": "509867806",
    "screen_name": "AiratAir"
  }
]

twurl "/1.1/search/tweets.json?q=%23london&count=100" | jq '[.statuses[]] | length'

-> 6

(i.e. put the statuses into a new array and count the length of that)

twurl -t -A "Content-Type: application/json" -d '{"id": "custom-543399461108469761","changes": [{ "op": "add", "tweet_id": "390897780949925889"},{ "op": "add", "tweet_id": "390853164611555329"}]}' "/1.1/collections/entries/curate.json"

‼️ REQUIRES twurl 0.9.3 (or Content-Type will be ignored -> fail)

Get a Tweet with (poll data expansion) from the Twitter Developer Labs Tweets endpoint (supports multiple IDs)

twurl "/labs/1/tweets?ids=1155833627743272960&format=detailed&expansions=author_id,attachments.poll_ids"

{
  "data": [
    {
      "id": "1155833627743272960",
      "created_at": "2019-07-29T13:33:05.000Z",
      "text": "If you're a developer using the Twitter API, what is you favourite tool to use to test endpoints? 🛠💻",
      "author_id": "786491",
      "attachments": {
        "poll_ids": [
          "1155833626535387136"
        ]
      },
      "stats": {
        "retweet_count": 2,
        "reply_count": 4,
        "like_count": 5,
        "quote_count": 1
      },
      "possibly_sensitive": false,
      "lang": "en",
      "source": "<a href=\"https://mobile.twitter.com\" rel=\"nofollow\">Twitter Web App</a>",
      "format": "detailed"
    }
  ],
  "includes": {
    "users": [
      {
        "id": "786491",
        "created_at": "2007-02-21T15:14:48.000Z",
        "name": "Pipes",
        "username": "andypiper",
        "protected": false,
        "location": "London, UK, Europe",
        "url": "https://t.co/jHwGP7KDmZ",
        "description": "What's Next? Twitter DevRel - Code, community, and respect. Bi and Proud 🏳️‍🌈 (they/them) Avatar credit: @ntakayama  👏🏻",
        "verified": false,
        "entities": {
          "url": {
            "urls": [
              {
                "start": 0,
                "end": 23,
                "url": "https://t.co/jHwGP7KDmZ",
                "expanded_url": "https://about.me/andypiper",
                "display_url": "about.me/andypiper"
              }
            ]
          },
          "description": {
            "mentions": [
              {
                "start": 107,
                "end": 117,
                "username": "ntakayama"
              }
            ]
          }
        },
        "profile_image_url": "https://pbs.twimg.com/profile_images/1117090202957303808/I0obUxtA_normal.jpg",
        "stats": {
          "followers_count": 16214,
          "following_count": 3489,
          "tweet_count": 121735,
          "listed_count": 854
        },
        "most_recent_tweet_id": "1178302696929009664",
        "pinned_tweet_id": "1156181746817392640",
        "format": "detailed"
      }
    ],
    "polls": [
      {
        "id": "1155833626535387136",
        "options": [
          {
            "position": 1,
            "label": "twurl",
            "votes": 11
          },
          {
            "position": 2,
            "label": "Postman",
            "votes": 24
          },
          {
            "position": 3,
            "label": "Insomnia",
            "votes": 9
          },
          {
            "position": 4,
            "label": "I roll my own!",
            "votes": 10
          }
        ],
        "voting_status": "closed",
        "end_datetime": "2019-07-30T13:33:05.000Z",
        "duration_minutes": 1440
      }
    ]
  }
}

Get Tweet metrics for a Tweet that is owned by your auth'ed user (supports multiple IDs)

twurl "/labs/1/tweets/metrics/private?ids=1171768043497803776"

{
  "data": [
    {
      "tweet_id": "1171768043497803776",
      "tweet": {
        "like_count": 31,
        "retweet_count": 17,
        "quote_count": 1,
        "reply_count": 8,
        "impression_count": 34148
      }
    }
  ]
}

The Engagement API is a commercial API that enables access to data about Tweet engagement (impressions, etc). Your application key can be granted permission for these endpoints if you've purchased access to the API. Contact Twitter here to discuss access.

‼️ Note that you can only request information about Tweet IDs owned by the authenticating user account.

twurl -H data-api.twitter.com -A "Content-Type: application/json" -X POST "/insights/engagement/totals" -d '{"tweet_ids": ["908115328856621056","908279258480594944","908296740691947520"],"engagement_types":["impressions","engagements","favorites"],"groupings": {"grouping name": {"group_by": ["tweet.id","engagement.type"]}}}'

{
  "grouping name": {
    "908115328856621056": {
      "engagements": "16",
      "favorites": "5",
      "impressions": "556"
    },
    "908279258480594944": {
      "engagements": "25",
      "favorites": "4",
      "impressions": "278"
    },
    "908296740691947520": {
      "engagements": "3",
      "favorites": "1",
      "impressions": "48"
    }
  }
}

twurl "/1.1/users/show.json?screen_name=andypiper&exclude_status=true" | jq '. | {followers: .followers_count}'

{
  "followers": 14815
}

$ twurl -t -H stream.twitter.com -A "Accept-encoding: none" -d "follow=54931584" -X POST "/1.1/statuses/filter.json" | jq --unbuffered

‼️ use Accept-encoding: none to avoid gzipped data

‼️ must add params to -d since this is a POST

#!/bin/sh
# count members in Twitter list
twurl "/1.1/lists/show.json?slug=$1&owner_screen_name=andypiper" | jq '. | {count: .member_count}'
#!/bin/sh
# remove from Twitter list
twurl -q -d "screen_name=$2&slug=$1&owner_screen_name=andypiper" "/1.1/lists/members/destroy.json"
#!/bin/sh
#
# get an app-only auth token (Bearer Token)
# pass in consumer key and consumer secret as params
# expected JSON response of form {"access_token":"CCCCCCCC","token_type":"bearer"}
#
# remove | python -m json.tool if no Python installed (or, no need to prettify results)
#
curl -s -S -u "$1:$2" --data 'grant_type=client_credentials' 'https://api.twitter.com/oauth2/token' | python -m json.tool

twurl -H publish.twitter.com "/oembed?url=https://twitter.com/andypiper&limit=5"

{
  "url": "https://twitter.com/andypiper",
  "title": "",
  "html": "<a class=\"twitter-timeline\" data-tweet-limit=\"5\" href=\"https://twitter.com/andypiper\">Tweets by andypiper</a>\n<script async src=\"//platform.twitter.com/widgets.js\" charset=\"utf-8\"></script>",
  "width": null,
  "height": null,
  "type": "rich",
  "cache_age": "3153600000",
  "provider_name": "Twitter",
  "provider_url": "https://twitter.com",
  "version": "1.0"
}

Handy shell alias for making twurl output more readable...

jsonator () {
	ruby -rubygems -r pp -e 'require "json"; ARGF.each {|l| puts JSON.pretty_generate(JSON.parse(l))}'
}

jtwurl () {
	twurl $@ | jsonator
}

Two stages: create a welcome message, and assign it to the default rule.

‼️ the account must have "Receive Direct Messages from anyone" enabled in account settings or these calls will fail with an error.

twurl -A 'Content-type: application/json' /1.1/direct_messages/welcome_messages/new.json -d '{"name": "evil-welcome", "welcome_message": {"message_data": {"text": "Welcome! I am the evil version of @andypiper"}}}'

(take note of the id value in the response to use in the follow-up call)

twurl -A 'Content-type: application/json' -X POST /1.1/direct_messages/welcome_messages/rules/new.json -d '{"welcome_message_rule": {"welcome_message_id": "966308520869187588"}}'

twurl "/1.1/statuses/user_timeline.json?count=5" | jq '[.[] | { text: .text, source: .source, time: .created_at}]'

[
  {
    "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque interdum rutrum sodales. Nullam mattis fermen… https://t.co/kb5SVo14NR",
    "source": "<a href=\"http://wibble.org\" rel=\"nofollow\">simpletester123</a>",
    "time": "Wed Jul 26 10:12:53 +0000 2017"
  },
  {
    "text": "test https://t.co/76eZnhqKj4",
    "source": "<a href=\"http://wibble.org\" rel=\"nofollow\">simpletester123</a>",
    "time": "Wed Jul 26 10:11:44 +0000 2017"
  },
  {
    "text": "Well done @LastWeekTonight 👏🏻 https://t.co/DyHKpUKGy1",
    "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter for iPhone</a>",
    "time": "Mon Jul 24 19:19:16 +0000 2017"
  },
  {
    "text": "Test https://t.co/wBZ9lp9kCR",
    "source": "<a href=\"https://fabric.io\" rel=\"nofollow\">PhotoboothDemoiOS</a>",
    "time": "Fri Jun 30 12:25:50 +0000 2017"
  },
  {
    "text": "just a test\nlook at this\n😎 https://t.co/gYEcopskOR",
    "source": "<a href=\"https://fabric.io\" rel=\"nofollow\">PhotoboothDemoiOS</a>",
    "time": "Thu Jun 29 17:26:21 +0000 2017"
  }
]

Summarise a list of people:

twurl "/1.1/lists/members.json?slug=First-50&owner_screen_name=andypiper&count=5000&skip_status=1&include_entities=false" | jq '[.users[] | {id: .id_str, name: .name, screen_name: .screen_name}]' > peeps.json

Summarise a set of lists a user is a member of:

twurl "/1.1/lists/memberships.json?screen_name=andypiper&count=650" | jq '[.lists[] | {name: .name, description: .description, uri: .uri, people: .member_count, subscribers: .subscriber_count}]'

Get list of lists owned by a user twurl "/1.1/lists/list.json?screen_name=andypiper" | jq '.[] | {slug: .slug, id: .id}'

#!/bin/bash
# convert UNIX epoch to timestamp (useful for rate limit resets)
perl -e "print scalar(localtime($1)), \"\n\""

$ FILE=$(base64 unnamed.png)

$ echo $FILE // just to prove that worked...

$ twurl -X POST -d "image=$FILE" "/1.1/account/update_profile_image.json

twurl "/1.1/users/search.json?q=andypiper" | jq '[.[] | {id: .id_str, name: .name, screen_name: .screen_name}]'

[
  {
    "id": "786491",
    "name": "Pipes",
    "screen_name": "andypiper"
  },
  {
    "id": "77825609",
    "name": "Andy Piper",
    "screen_name": "andypiper8"
  },
  {
    "id": "455565439",
    "name": "Andrew St",
    "screen_name": "Andypip14"
  },
  {
    "id": "332796322",
    "name": "Andy Pipe",
    "screen_name": "andypipe38"
  },
  {
    "id": "424300699",
    "name": "Felipe Basto",
    "screen_name": "Andypipe98Basto"
  },
  {
    "id": "2894986614",
    "name": "Andy Piper",
    "screen_name": "AndyAndypiper"
  },
  {
    "id": "244145232",
    "name": "andres felipe",
    "screen_name": "andypipe7"
  },
  {
    "id": "427650744",
    "name": "andy pipe",
    "screen_name": "andypipe3"
  },
  {
    "id": "295296501",
    "name": "ANDRES FELIPE BOTINA",
    "screen_name": "andypipe2011"
  },
  {
    "id": "3119979580",
    "name": "andres cardenas",
    "screen_name": "andypipe1207"
  },
  {
    "id": "175910590",
    "name": "Anderson Felipe",
    "screen_name": "andypipe_"
  },
  {
    "id": "2215376413",
    "name": "andy.pip",
    "screen_name": "andypip1"
  },
  {
    "id": "156437723",
    "name": "andres felipe pinzon",
    "screen_name": "andypipe29"
  },
  {
    "id": "4774876672",
    "name": "andy",
    "screen_name": "andypip69"
  },
  {
    "id": "268190008",
    "name": "Philip Piper",
    "screen_name": "andypiper3971"
  },
  {
    "id": "915079999",
    "name": "Andy Piper",
    "screen_name": "andypiper1985"
  },
  {
    "id": "1388227916",
    "name": "andres felipe garzon",
    "screen_name": "andypipe22"
  },
  {
    "id": "1570721150",
    "name": "AndyPipe",
    "screen_name": "daniandi2002"
  },
  {
    "id": "180432443",
    "name": "Andres Felipe",
    "screen_name": "andypipe206"
  },
  {
    "id": "236309993",
    "name": "AndyPipes",
    "screen_name": "xcellxx"
  }
]
// use the twitter-text library to parse and validate a Tweet string
// updated for twitter-text 2.0 new semantics
// npm install twitter-text
// node ./validate-tweet.js
var twitter = require('twitter-text');
var tweet = "123 @andypiper 123456 @mauropiano #winning $TWTR 8901789zz #gnip 0123456789... http://blog.rust-lang.org/2016/05/26/Rust-1.9.html more at http://localhost:3000/notes/595 @juandoming Dallas teachers improved student performance by 20% w/mobile video messaging. Would this be useful to you? stars.now wibble wobble testing #pipes";
//var tweet = "@juandoming Dallas teachers improved student performance by 20% w/mobile video messaging. Would this be useful to you? stars.now"
//var tweet = "testing testing #blog https://andypiper.co.uk"
var parsed = twitter.parseTweet(tweet);
var chars = parsed.weightedLength;
var valid = parsed.valid;
var usernames = twitter.extractMentions(tweet);
var urls = twitter.extractUrls(tweet);
var hash = twitter.extractHashtags(tweet);
var cash = twitter.extractCashtags(tweet);
console.log("text: " + tweet + "\n");
console.log("validity: " + valid + "\n");
console.log("length: " + chars);
console.log("users: " + usernames);
console.log("links: " + urls);
console.log("hashtags: " + hash);
console.log("cashtags: " + cash);
@altanner

This comment has been minimized.

Copy link

altanner commented Aug 21, 2018

Thanks for these. Can twurl be used to recover tweets from a certain length of time ago, for a specified user? For example, taking all tweets from a user's timeline from the last day, or the last seven days?

@alhadif1

This comment has been minimized.

Copy link

alhadif1 commented Oct 28, 2018

شكرا لكم
اريد حلا لا اضافه المتابعين أو إلغاء المتابعين وشكرا

@andypiper

This comment has been minimized.

Copy link
Owner Author

andypiper commented Dec 7, 2018

@altanner you cannot "recover" Tweets, but yes, you could use twurl to call the timeline API to fetch up to 3200 Tweets from a public timeline.

@MooseCodee

This comment has been minimized.

Copy link

MooseCodee commented Sep 22, 2019

Thank you for these Andy, I'd like some advice on how I use this API to GET all my followers displayed on the website I'm working on and then filter them based on their name, BIO etc?

Your thoughts are appreciated

@andypiper

This comment has been minimized.

Copy link
Owner Author

andypiper commented Sep 23, 2019

I probably would not use twurl for this, as it is a command line tool intended more for scripting and interactive usage. Depending on your coding language of choice, I'd pick a relevant API library wrapper, and then use the GET followers/list endpoint. Depending on the number of followers you have, you might need to tweak the count parameter up to the maximum (200), and implement paging. Finally, the filtering would be down to some custom coding based on the chosen fields from the returned user object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.