Grab oauth token for use with Nitter (requires Twitter account)

twitter_oauth.sh

#!/bin/bash
# Grab oauth token for use with Nitter (requires Twitter account). 
# results: {"oauth_token":"xxxxxxxxxx-xxxxxxxxx","oauth_token_secret":"xxxxxxxxxxxxxxxxxxxxx"}
# verified: 2024-11-14

username=""
password=""

if [[ -z "$username" || -z "$password" ]]; then
  echo "needs username and password"
  exit 1
fi

bearer_token='AAAAAAAAAAAAAAAAAAAAAFXzAwAAAAAAMHCxpeSDG1gLNLghVe8d74hl6k4%3DRUMF4xAQLsbeBhTSRrCiQpJtxoGWeyHrDb5te2jpGskWDFW82F'
guest_token=$(curl -s -XPOST https://api.twitter.com/1.1/guest/activate.json -H "Authorization: Bearer ${bearer_token}" -d "grant_type=client_credentials" | jq -r '.guest_token')
base_url='https://api.twitter.com/1.1/onboarding/task.json'
header=(-H "Authorization: Bearer ${bearer_token}" -H "User-Agent: TwitterAndroid/10.21.0-release.0" -H "X-Twitter-Active-User: yes" -H "Content-Type: application/json" -H "X-Guest-Token: ${guest_token}")

# start flow
flow_1=$(curl -si -XPOST "${base_url}?flow_name=login&api_version=1&known_device_token=&sim_country_code=us" "${header[@]}" \
  -d '{"flow_token": null, "input_flow_data": {"country_code": null, "flow_context": {"referrer_context": {"referral_details": "utm_source=google-play&utm_medium=organic", "referrer_url": ""}, "start_location": {"location": "deeplink"}}, "requested_variant": null, "target_user_id": 0}}'
)

# get 'att', now needed in headers, and 'flow_token' from flow_1
att=$(sed -En 's/^att: (.*)\r/\1/p' <<< "${flow_1}")
flow_token=$(sed -n '$p' <<< "${flow_1}" | jq -r .flow_token)

# username
token_2=$(curl -s -XPOST "${base_url}" -H "att: ${att}" "${header[@]}" \
  -d '{"flow_token": "'"${flow_token}"'", "subtask_inputs": [{"enter_text": {"suggestion_id": null, "text": "'"${username}"'", "link": "next_link"}, "subtask_id": "LoginEnterUserIdentifier"}]}' | jq -r .flow_token)

# password flow and print oauth_token and secret
curl -s -XPOST "${base_url}" -H "att: ${att}" "${header[@]}" \
  -d '{"flow_token": "'"${token_2}"'", "subtask_inputs": [{"enter_password": {"password": "'"${password}"'", "link": "next_link"}, "subtask_id": "LoginEnterPassword"}]}' |
  jq -c '.subtasks[0]|if(.open_account) then {oauth_token: .open_account.oauth_token, oauth_token_secret: .open_account.oauth_token_secret} else empty end'

(This is for personal use, accounts are heavily throttled)

Write output to guest_accounts.jsonl

Comment out line 205 in src/auth.nim for now if you have an old account, to make it valid over 30 days.

#accountPool.keepItIf(not it.hasExpired)

...

[accounts] Parsing JSONL guest accounts file: ./guest_accounts.jsonl
[accounts] Successfully added 2 valid accounts.
Starting Nitter at http://shitter

Re-implemented this in Python, so explicitly calling out some things for anyone doing the same:

• the curl -si when fetching task_1 means we're dumping headers, so look for att in response.headers
• in the final output (containing OAuth tokens), the subtask_id is LoginSuccessSubtask rather than OpenAccount, as it is for nitter#983 "guest accounts"

And for usage, a comment from nitter issue #1156:

the account gets locked if you try to paginate more than a couple of pages in search

When I run this script on macos 11.2.3, I am able to retrieve att and flow_token, however, token_2 and token_3 are empty. Am I the only one like this?

Numerous people are having issues with this. I had to use it today and it still works for me. You can try to see where it fails by adding the -v (verbose) option to curl, remove the jq pipe command and echo token_2 after the fetch command to try and pinpoint where it fails. There may be a email verification check too, this script doesn't handle that...

Try these other options if you can't get this to work:

• zedeus/nitter#983 (comment)
• https://gist.github.com/cmj/c45c46d9dd11fa2d4ba4de55c50598e6 <- has email element but unable to verify on my end

@cmj Thanks. I have tried both the JavaScript and Python versions to get account information and using curl manually step by step. All of them worked perfectly. I am just confused as to why this script isn't working for me. I also try to run this on Github Action and get the same results.

Empty reply from server as TLSv1.3 (IN), TLS alert, close notify (256), i have no idea how to fix it.

* We are completely uploaded and fine
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* TLSv1.3 (IN), TLS alert, close notify (256):
{ [2 bytes data]
* HTTP/2 stream 1 was not closed cleanly before end of the underlying stream
} [5 bytes data]
* Closing connection 0
} [5 bytes data]
* TLSv1.3 (OUT), TLS alert, close notify (256):
} [2 bytes data]

@LarchLiu I took another look at this... The issue was there were control characters at the end of the att header that mangled the request, so col -b now strips them accounting for the carriage return ^M (\r) fixes this.

Cleaned up other elements too so it should work everywhere.

codespaces-564d0b ➜ $ ./twitter_oauth.sh
{"oauth_token":"169xxxxxxxxxxx-xxxxxxxxxxx","oauth_token_secret":"xxxxxxxxxxxx"}
codespaces-564d0b ➜ $

and,

@cmj Awesome! It works, thank you so much. 🎉

Is this still working? The script doesn't seem to output anything?

@flikites Quickly looking this over, it's failing on the password flow. Something changed and perhaps they hash the password before sending now? Not sure.

It's been a number of months since I understood the process; I'll try and poke around more this weekend.

Thanks for the heads up!

Thank you @cmj Looking forward to getting my nitter instance rolling if at all possible. I appreciate you taking the time to look.

So after throwing a bunch of extra steps in to recreate the login process, It looks as if they just flat out removed the AccountDuplicationCheck/check_logged_in_account subtask which contained oauth_token and secret.

All oauth tokens created before the removal are still working and probably will indefinitely.

It now seems to me we will have to revert back to version from @PrivacyDevel that utilized the cookie headers (x-csrf-token and auth_token.) I believe at one point we had both cookies option and oauth working together.

This might take some time but will work, only downside is there isn't multiple account support for using cookies at the moment.

---

This has every login step I can think of and it was quickly thrown together so..
https://github.com/cmj/twitter-tools/blob/c44d9568f14e21194ba06c7ac78d26c63213275e/twitter_oauth.sh
cmj/twitter-tools#1 (comment)

Files are there for reference; this process no longer works.

@flikites I just started a branch that uses cookies instead of oauth tokens. It only allows for 1 account, but I think most people were using 1 to begin with anyways.

Seems to work fine so far. Only thing that's not is the media tab, as far as I can tell.

Instructions to try it out: https://github.com/cmj/nitter/wiki/Install

Make sure to checkout the branch: git checkout cookie_header

I am using the https://github.com/sekai-soft/guide-nitter-self-hosting configuration that works perfect, but the inly problem that I have is that I can't add more than one account. I get rate limited easly

They are also using the same process to get old oauth tokens.

https://github.com/sekai-soft/guide-nitter-self-hosting/blob/4404ad05fb99b94c5d02bec2b53bf1076f22dda8/nitter-auth/nitter-auth.py#L149-L150 (I just tested the script; also fails)

I believe this stopped working sometime at the beginning of August, based on various github issues searches.

Here is the latest with some findings in comments:

https://gist.github.com/cmj/041bcc04f1a1ae7624a01ecd72d7a108

You can see it's a successful attempt to finish the loginFlow, but it now fails to return 'open_account' subtask. So from my understanding, you'll no longer be able to obtain oauth_token & oauth_token_secret. Existing tokens you've created will still work (I have some from years ago.)

This is why I've been toying with just using cookie authentication option. I've been running it exclusively for 2 days and seems to be working fine so far. Mostly just been tearing stuff out that's not needed so it will have to be cleaned up at some point.

---

So, for now, you cannot add anymore new Twitter accounts to Nitter. Unless someone finds the magical formula in getting those tokens, this seems to be the only thing useful you can extract from the loginFlow https://github.com/cmj/nitter/blob/cookie_header/twitter_auth_token.sh

Thanks @cmj

Gonna test this out in a few.

Update: Working. Thank You!

@flikites The 'cookie' branch? Glad to hear, I've been using it exclusively for about a week and seems to be fine so far. There are a bunch of backend things that need to be polished but as a PoC it's working well.

Thanks for the feedback!

@flikites The 'cookie' branch? Glad to hear, I've been using it exclusively for about a week and seems to be fine so far. There are a bunch of backend things that need to be polished but as a PoC it's working well.

Thanks for the feedback!

Yes! The 'cookie_header' branch. Working good on my end as well since I've had it up.

Thanks so much for the cookie header branch! Is it a big lift to make it work for multiple cookie headers?

@mrdev1337 Yeah that's ultimately the goal. Obviously the framework is there to handle multiple accounts, I just disabled it all to see if it worked fine with just one.

One would have to refactor in all the ratelimits too as I think they might differ than before. I don't have a comprehensive understanding of account management yet, but it's something I'm willing to look at.

Nitter is dead for new multiple-account installs otherwise.

Made some modifications and this script seems to be working again.