Twitter: How to archive your following/followers data (usernames, etc)

twitter-archive-following-followers.md

Twitter allows users to download parts of their data, see How to download your Twitter archive.

But what's not included in that data dump is the usernames/handles of the people that you follow or are following you. All you get is account IDs which is just an internal number and so a bit useless when it comes to archival.

Here's a way to get that data (you need to know how to run stuff in the terminal):

1. Go to your Twitter profile in a desktop browser (Firefox or Chrome)
2. Right click on page → Inspect → Network tab
3. Click on the Following link (e.g. https://twitter.com/{yourusername}/following)
4. Scroll through the entire list of following users (e.g. keep pressing page down, space or scroll)
5. In the filter/search bar of the Network tab: type Following? to only get the requests we're interested in
6. Right click on requests → Save All As HAR or Save all as HAR with content; Name it following.har

Then, in a terminal, with jq installed, run:

jq -r '.log.entries[] | .response.content.text | fromjson? | .data.user.result.timeline.timeline.instructions[-1].entries[] | .content.itemContent.user_results.result | values | [.rest_id, .legacy.screen_name, .legacy.name, .legacy.description] | @csv' following.har > following.csv

That will give you the ID, username, display name and description in a CSV file.

If you want to archive all the data that Twitter returns, run this to get a JSON stream:

jq -r '.log.entries[] | .response.content.text | fromjson? | .data.user.result.timeline.timeline.instructions[-1].entries[] | .content.itemContent.user_results.result | values' following.har > following.json

If you want to export your followers too, clear the Network requests and then repeat the steps but use the Followers link, filter requests by Followers? and use different filenames. The rest is the same.

(Tweet about it here: https://twitter.com/niborst/status/1589883662048579584)

Thanks so much for this! Just a note that these instructions are for Firefox.

No worries :). Ah yeah. I think for Chrome they should be similar enough, right (I hope the jq doesn't have to change)?

I couldn't get it to work with the Chrome har file so it might need some tweaking, but it was no hassle since Firefox is my browser of choice.

@wonderingwombat Thanks! The problem was that Chrome includes all requests, not only the filtered ones. I've updated the jq command to handle that now 🎉.

OMG thank you very much!!! Just what I needed, worked perfectly!

Hey, any help with this?

@quiet69 Run the command in WSL, not Windows

Thank you!

Data format seems to have changed a bit and I couldn't be bothered to figure out how to fix the command, so instead I did:

jq -r '.log.entries[] | .response.content.text | fromjson?' twitter.com.har > followers.json

I then used my code editor's search features to replace all the lines that did not contain screen_name. Good enough for me.