Skip to content

Instantly share code, notes, and snippets.

View igorbrigadir's full-sized avatar

Igor Brigadir igorbrigadir

View GitHub Profile
@igorbrigadir
igorbrigadir / flattened.jsonl
Created July 29, 2021 16:14
Twitter API original Vs twarc flatten output
{"text": "RT @qeensdale: But it's #Boris the fool who is to blame for the latest troubles in Northern Ireland his bloody #Brexit deal.\nBest if he fal\u2026", "lang": "en", "possibly_sensitive": false, "conversation_id": "1384644560446447617", "source": "Twitter for Android", "created_at": "2021-04-20T23:06:03.000Z", "referenced_tweets": [{"type": "retweeted", "id": "1384551429306781700", "text": "But it's #Boris the fool who is to blame for the latest troubles in Northern Ireland his bloody #Brexit deal.\nBest if he falls on his sword. \n\nBBC News - Brexit: Boris Johnson promises to end GB-NI trade barriers\nhttps://t.co/uTbvahkvcz", "lang": "en", "entities": {"urls": [{"start": 213, "end": 236, "url": "https://t.co/uTbvahkvcz", "expanded_url": "https://www.bbc.co.uk/news/uk-northern-ireland-56777985", "display_url": "bbc.co.uk/news/uk-northe\u2026"}], "hashtags": [{"start": 9, "end": 15, "tag": "Boris"}, {"start": 96, "end": 103, "tag": "Brexit"}], "annotations": [{"start": 68, "end": 83, "probability": 0.
@igorbrigadir
igorbrigadir / news_domain_list.json
Last active January 31, 2024 22:00
Twitter List of News Domains
[
"francemusique.fr",
"sumikai.com",
"yadokari.net",
"epoca.globo.com",
"airedesantafe.com.ar",
"nemgym.com",
"numerama.com",
"analgeria.com",
"bobiannews.com",
@igorbrigadir
igorbrigadir / mutuals.sh
Last active March 30, 2024 09:35
Twitter Mutuals CSV with twarc
#!/bin/bash
# run this like ./mutuals.sh screen_name
# Get all the accounts you follow
twarc2 followers "$1" "$1_followers.jsonl"
# Get all the accounts that follow you
twarc2 following "$1" "$1_following.jsonl"
# Get mutuals ids
This file has been truncated, but you can view the full file.
{
"primary_text": {
"text": "Follow your favorite Topics",
"entities": []
},
"secondary_text": {
"text": "You’ll see top Tweets about them in your timeline. Don’t see your favorite Topics yet? New Topics are added every week.",
"entities": []
},
"top_category_ids": [
@igorbrigadir
igorbrigadir / MariamBarghouti.json
Created May 11, 2021 21:52
@MariamBarghouti Media Policy violation Twitter API Response
{
"data": [
{
"entities": {
"annotations": [
{
"start": 33,
"end": 39,
"probability": 0.9758,
"type": "Place",
C:\Users\USER>curl -v -H "Authorization: Bearer
AAA...zzz"
"
https://api.twitter.com/2/tweets/search/recent?expansions=author_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id%2Centities.mentions.username%2Cattachments.poll_ids%2Cattachments.media_keys%2Cgeo.place_id&user.fields=created_at%2Cdescription%2Centities%2Cid%2Clocation%2Cname%2Cpinned_tweet_id%2Cprofile_image_url%2Cprotected%2Cpublic_metrics%2Curl%2Cusername%2Cverified%2Cwithheld&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Ctext%2Cpossibly_sensitive%2Creferenced_tweets%2Creply_settings%2Csource%2Cwithheld&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics&poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&place.fields=contained_within%2Ccountry%2Ccountry_code%2Cfull_name%2Cgeo%2Cid%2Cname%2Cplace_type&max_results=10&query=end
@igorbrigadir
igorbrigadir / response-expanded.jsonl
Created March 2, 2021 11:32
Twitter v2 Full API response and expanded tweets
{"conversation_id": "1365618709222600705", "text": "RT @tradehunter305: While checking into my hotel, I seen this light skin dude behind me. We ended up on the same floor, I asked him if he w\u2026", "reply_settings": "everyone", "possibly_sensitive": false, "source": "Twitter for iPhone", "public_metrics": {"retweet_count": 1256, "reply_count": 0, "like_count": 0, "quote_count": 0}, "lang": "en", "referenced_tweets": [{"type": "retweeted", "id": "1353880904188506112", "entities": {"urls": [{"start": 274, "end": 297, "url": "https://t.co/9CovM6hF9P", "expanded_url": "https://twitter.com/tradehunter305/status/1353880904188506112/video/1", "display_url": "pic.twitter.com/9CovM6hF9P"}], "hashtags": [{"start": 228, "end": 243, "tag": "tradehunter305"}, {"start": 244, "end": 256, "tag": "miamifreaks"}], "annotations": [{"start": 134, "end": 141, "probability": 0.5495, "type": "Person", "normalized_text": "Ofocurse"}], "mentions": [{"start": 178, "end": 191, "username": "_BDhandsome1"}, {"start": 257, "end": 273, "
@igorbrigadir
igorbrigadir / BookmarkAPI_en.md
Created August 3, 2020 12:46 — forked from stepney141/BookmarkAPI_en.md
WIP: Twitter Undocumented Endpoints for Bookmark

Twitter Undocumented Bookmark API (WIP)

I found out the endpoints for bookmark with Chrome Developer Tools: GET timeline/bookmark, POST bookmark/entries/remove, POST bookmark/entries/remove. The rate limits below are values returned by an official endpoint GET application/rate_limit_status.

This document is still a work in progress because I got stuck in GET timeline/bookmark. Please let me know if you find how to use it.

Notes

  • It is necessary that x-csrf-token in a request header and ct0 in a cookie are the same value. Twitter uses them to avoid CSRF attacks. I recommend that you extract the values from your browsers.
  • All of the endpoints requires OAuth2 Authorizations. Note that they refuse OAuth2 Bearer tokens obtained from POST oauth2/token.
  • You can easily reach the rate limit and get HTTP 429 Error (too many requests), so you should be careful about how many requests you send.
@igorbrigadir
igorbrigadir / cuda_docker_dev.md
Last active July 30, 2020 14:13
Working with Notebooks on GPUs with CUDA

Working with Notebooks on GPUs with CUDA

Like a lot of people, i prototype things in notebooks. When it comes to working with GPUs, a common problem i've had to deal with over and over is keeping my environment in delicate balance.

A Stable arrangement I ended up using:

@igorbrigadir
igorbrigadir / pandas_multiprocess.ipynb
Created February 24, 2020 11:48
Resumable, Multiprocessing apply for Pandas Dataframes
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.