Skip to content

Instantly share code, notes, and snippets.

@stepney141

stepney141/BookmarkAPI_en.md

Last active April 4, 2023 01:45
Show Gist options
  • Star 74 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save stepney141/c161a83f02c42e161c905249733b9225 to your computer and use it in GitHub Desktop.
Save stepney141/c161a83f02c42e161c905249733b9225 to your computer and use it in GitHub Desktop.
Download ZIP
(DEPRECATED) Twitter Undocumented Endpoints for Bookmark
Raw
BookmarkAPI_en.md

Twitter Undocumented Bookmark API (DEPRECATED)

Update: 2022-03-25

Twitter has released the official API v2 endpoint for the bookmark feature. https://twittercommunity.com/t/build-with-bookmarks-on-the-twitter-api-v2/168804/

The following descriptions are or will soon be no longer useful; I suggest using the new official API.

I found out the endpoints for bookmark with Chrome Developer Tools: GET timeline/bookmark, POST bookmark/entries/remove, POST bookmark/entries/remove. The rate limits below are values returned by an official endpoint GET application/rate_limit_status.

This document is still a work in progress because I got stuck in GET timeline/bookmark. Please let me know if you find how to use it.

Notes

  • It is necessary that x-csrf-token in a request header and ct0 in a cookie are the same value. Twitter uses them to avoid CSRF attacks. I recommend that you extract the values from your browsers.
  • All of the endpoints requires OAuth2 Authorizations. Note that they refuse OAuth2 Bearer tokens obtained from POST oauth2/token.
  • You can easily reach the rate limit and get HTTP 429 Error (too many requests), so you should be careful about how many requests you send.
  • I have heard that someone said that "GET timeline/bookmark" returned HTTP 403 Error even though OAuth authentication succeeded. Maybe the endpoint refuses mechanical accesses.
  • In some cases, perhaps it is better to use the official TweetDeck Collection API instead of the undocumented and uncertain API.
  • cf: https://github.com/geekodour/twitmarks/ / https://github.com/acorn/twitter-bookmarks-search (It seems the developers understand how to use the endpoints)
Raw
BookmarkAPI_ja.md

Twitterブックマークの非公開API（調査未完了）

Update: 2022-03-25

Twitter API v2にて、新たにブックマーク操作用の公式APIがリリースされました。 https://twittercommunity.com/t/build-with-bookmarks-on-the-twitter-api-v2/168804/

下記の調査内容は既に使えなくなっているか、または近い将来使えなくなる可能性があります。公式のAPIを使用することをオススメいたします。

Chromeのデベロッパーツールをこねくり回して見つけたエンドポイント（利用方法要調査）

ドキュメントの日本語版は"GET timeline/bookmark"の解析が完了したら書きます。
※リクエスト制限は公式エンドポイントの"GET application/rate_limit_status"で返された値です。

その他情報

  • リクエスト制限を超えて叩くとHTTP 429 Error(Too many requests)を返す模様
  • 「"GET timeline/bookmark"が403 Errorを返してきた」「認証が通った上で弾かれているように見える」という趣旨の情報あり
  • わざわざこれ使わなくてもTweetDeckのcollection機能で代用可能かも（TD以外では使用できないが）
  • cf: https://github.com/geekodour/twitmarks/ （先駆者による解析内容が使われたOSS、要ソース確認）
Raw
EntriesAdd_en.md

POST bookmark/entries/add

Adds the Tweet specified by the tweet_id parameter to the users' bookmarks.

The API requires an OAuth2 Authorization. Note that it refuses Bearer tokens obtained from POST oauth2/token.

Resource URL

https://api.twitter.com/1.1/bookmark/entries/add.json

Resource Information
Response formats JSON
Requires authentication? Yes
Rate limited? Yes
Requests / 15-min window 180

Parameters

Name Required Description Default Value Example
tweet_id required The numerical ID of the Tweet to add to bookmark. 1234567890
tweet_mode optional Valid request values are compat and extended, which give compatibility mode and extended mode, respectively for Tweets that contain over 140 characters. extended

Example Request

POST /1.1/bookmark/entries/remove.json HTTP/1.1
Host: api.twitter.com
x-csrf-token: c4e034 ... ff317c
Authorization: Bearer AAAA%2FAAA%3DAAAAAAAA
cookie: auth_token=2ff2dfe3 ... 57bbd4ca; ct0=c4e034 ... ff317c
Content-Type: application/x-www-form-urlencoded

tweet_id=1234567890&tweet_mode=extended

Note that x-csrf-token in a request header and ct0 in a cookie have to be the same value. Twitter uses them to avoid CSRF attacks. I recommend you extract the values from your browsers.

Example Response

{
    "objects": {},
    "response": {
        "errors": []
    }
}
Raw
EntriesRemove_en.md

POST bookmark/entries/remove

Removes the Tweet specified by the tweet_id parameter from the users' bookmarks.

The API requires an OAuth2 Authorization. Note that it refuses Bearer tokens obtained from POST oauth2/token.

Resource URL

https://api.twitter.com/1.1/bookmark/entries/remove.json

Resource Information
Response formats JSON
Requires authentication? Yes
Rate limited? Yes
Requests / 15-min window 180

Parameters

Name Required Description Default Value Example
tweet_id required The numerical ID of the Tweet to remove from bookmark. 1234567890
tweet_mode optional Valid request values are compat and extended, which give compatibility mode and extended mode, respectively for Tweets that contain over 140 characters. extended

Example Request

POST /1.1/bookmark/entries/remove.json HTTP/1.1
Host: api.twitter.com
x-csrf-token: c4e034 ... ff317c
Authorization: Bearer AAAA%2FAAA%3DAAAAAAAA
cookie: auth_token=2ff2dfe3 ... 57bbd4ca; ct0=c4e034 ... ff317c
Content-Type: application/x-www-form-urlencoded

tweet_id=1234567890&tweet_mode=extended

Note that x-csrf-token in a request header and ct0 in a cookie have to be the same value. Twitter uses them to avoid CSRF attacks. I recommend you extract the values from your browsers.

Example Response

{
    "objects": {},
    "response": {
        "errors": []
    }
}
Raw
TimelineBookmark_en.md

GET timeline/bookmark (WIP)

Resource URL

https://api.twitter.com/2/timeline/bookmark.json

Resource Information
Response formats JSON
Requires authentication? Yes
Rate limited? Yes
Requests / 15-min window 1000

Parameters

Name Required Description Default Value Example

Example Request

Example Response

@stepney141
Copy link
Author

stepney141 commented Apr 12, 2020 •
edited

I succeeded in requesting POST /1.1/bookmark/entries/add.json and POST /1.1/bookmark/entries/remove.json with Postman.

A successful HTTP request is like this:

POST /1.1/bookmark/entries/add.json HTTP/1.1
Host: api.twitter.com
x-csrf-token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Authorization: Bearer AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%2FAAAAAAAAAAAA
                      AAAAAAAA%3DAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
cookie: auth_token=BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB; ct0=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Content-Type: application/x-www-form-urlencoded

tweet_id=1234567891011121314&tweet_mode=extended

(The tokens and tweet_id in the above are example values)
x-csrf-token and ct0 have the same value.
A request for remove.json also successfully works with the same header values.
Note that we have to extract x-csrf-token, auth_token, ct0, and Bearer token in Authorizationout of browsers each time when we write requests.
I'm planning to develop a browser extension that makes the process easier.

A successful response body is like this:

{
    "objects": {},
    "response": {
        "errors": []
    }
}

Anyway, I finished my work for the add/remove endpoints, so I'll concentrate on /2/timeline/bookmark.json.

@jborichevskiy
Copy link

jborichevskiy commented Apr 13, 2020 •
edited

Glad to hear it was helpful!

As far as 2/timeline/bookmark.json goes, here is the code that was working for me last weekend (where working = successfully looping through 7 times to fetch all my bookmarks). It is no longer working today (second request fails with a 400 and {'errors': [{'code': 214, 'message': 'Bad request.'}]}). Here's my testing Python script:

def pull_bookmarks(headers, cookies):
    url = "https://api.twitter.com/2/timeline/bookmark.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_composer_source=true&include_ext_alt_text=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&count=20&ext=mediaStats%2ChighlightedLabel%2CcameraMoment"

    def extract_cursor(response):
        try:
            return response.json()['timeline']['instructions'][0]['addEntries']['entries'][-1]['content']['operation']['cursor']['value']
        except KeyError as e: 
            return None

    initial = True
    while True:
        if initial:
            response = api_request(url, headers, cookies)
            for tweet in response.json()['globalObjects']['tweets'].values():
                print(tweet['created_at'], tweet['id_str'], tweet['full_text'][:50])

            initial = False
        else:
            sleep(5)  # Apparently 4 is too fast and 6 is too slow... idk wtf is going on
            
            cursor_value = extract_cursor(response) 
            if cursor_value is None:
                break

            response = api_request(url + f'&cursor={cursor_value}', headers, cookies)
            print(response.status_code)

            if response.status_code != 200:
                break

            for tweet in response.json()['globalObjects']['tweets'].values():
                print(tweet['created_at'], tweet['id_str'], tweet['full_text'][:50])

I highly doubt they changed their API this week, so I suspect the problem is with interaction between my cookies and the cursor. Or earth's magnetic field. Who knows, really.

Hoping to continue poking at this next weekend, so will update if I get it working!

@thadk
Copy link

thadk commented Jul 24, 2020

I think this also supports the bookmarks.json API https://github.com/acorn/twitter-bookmarks-search

@stepney141
Copy link
Author

@thadk I didn't know the extension supports the bookmark api. I'll add it to my documents and try to examine it. Thank you very much!

@thadk
Copy link

thadk commented Jul 25, 2020 •
edited

I did npm install/npm run build the latest version, added it to Chrome to try the latest version on GitHub. Maybe it only gets and searches the latest page of bookmarks, but it could give an idea. It seemed to go back about 3 months through about 500 tweets.

@Kenan7
Copy link

Kenan7 commented May 2, 2021

does this work right now?

@stepney141

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment