Skip to content

Instantly share code, notes, and snippets.

View phonedude's full-sized avatar

Michael L. Nelson phonedude

23 followers · 0 following
View GitHub Profile
@phonedude
phonedude / wsdl-pubs-2023.md
Last active March 4, 2024 14:23
@WebSciDL's 2023 Publications
Collapse/expand the list of WSDL's 2023 publications

Web Science and Digital Libraries Research Group 2023 Publications

  1. B Farrow, S Jayarathna, A Serverless Electroencephalogram Data Retrieval and Preprocessing Framework, 2023 IEEE 24th International Conference on Information Reuse and Integration …, 2023.

  2. K Pineda, B Mahanama, V Ashok, S Jayarathna, Gaze-Augmented Drone Navigation, Proceedings of the Augmented Humans International Conference 2023, 363-366, 2023.

  3. SM Jones, M Klein, MC Weigle, ML Nelson,
@phonedude
phonedude / onwardchristiancounseling-shadowban.txt
Created November 9, 2023 00:03
Shadowbanned / ghostbanned #WebArchiveWednesday thread about www.onwardchristiancounseling.org
https://twitter.com/phonedude_mln/status/1722322426321568151
https://twitter.com/phonedude_mln/status/1722322429391880389
https://twitter.com/phonedude_mln/status/1722322432546017595
https://twitter.com/phonedude_mln/status/1722322435284811810
https://twitter.com/phonedude_mln/status/1722322438539632657
https://twitter.com/phonedude_mln/status/1722322442331279497
https://twitter.com/phonedude_mln/status/1722322446223593606
https://twitter.com/phonedude_mln/status/1722322449910386854
https://twitter.com/phonedude_mln/status/1722322452431118821
https://twitter.com/phonedude_mln/status/1722322455140635108
@phonedude
phonedude / tco-test-with-headers
Created September 18, 2023 16:49
replicating t.co delays as per: https://twitter.com/themarkup/status/1702744291263599021
mln-phonedude:~ mln$ more tco-test-with-headers
#!/bin/tcsh -x
time curl -i -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://twitter.com/notifications' -H 'Connection: keep-alive' https://t.co/SLUCgW6Pdo > /dev/null
time curl -i -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://twitter.com/notifications' -H 'Connection: keep-alive' https://t.co/JKnTpPpuKB > /dev/null
time curl -i -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://twitter.com/notifications' -H 'Connection: keep-alive' https://t.co/HH1DUkOUqn > /dev/null
time curl -i -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:78.0)
@phonedude
phonedude / atlanticofr-2023-06-24.txt
Created June 28, 2023 18:14
English transcript of https://atlantico.fr/article/decryptage/notre-memoire-numerique-collective-est-elle-en-train-de-disparaitre
Sky Blog which once was the leader of the blogs in France is going to shut down mid August. How does this exemplify the risk of losing our internet memory ?
It is indeed very concerning. This site holds significant cultural and technological importance from the time it was introduced, although blogging has since been overshadowed by social media platforms like Facebook, Twitter, Instagram, and others. I understand that it's not the same as it was when it first came out in 2002 or whenever it was.
Now, there is some precedent for this. Do you remember the Geocities site in the US? It was quite popular in the 90s and allowed users to create their own websites, although it wasn't exactly a blog platform. It enjoyed popularity until it was eventually sold to Yahoo, who struggled to find a clear direction for it and eventually shut it down. Consequently, this time capsule of the mid to late 90s mostly disappeared. While a few sites have recreated Geocities, and the Internet Archive has archived a significant
@phonedude
phonedude / wsdl-pubs-2022.md
Last active January 26, 2023 16:28
@WebSciDL's 2022 Publications
Collapse/expand the list of WSDL's 2022 publications

Web Science and Digital Libraries Research Group 2022 Publications

  1. X Wei, J Wu, K Ajayi, D Oyen, Visual descriptor extraction from patent figure captions: a case study of data efficiency between BiLSTM and transformer, Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries, 1-5, 2022.

  2. S Rajtmajer, C Griffin, J Wu, R Fraleigh, L Balaji, A Squicciarini, ..., A synthetic prediction market for estimating confidence in published work, Proceedings of the AAAI Conference on Artificial Intelligence 36 (11), 13218 …, 2022.

  3. J Owens, G Jayawa
@phonedude
phonedude / grab-twitter-video-from-ia.txt
Created December 12, 2022 22:53
The Twitter DM video files have been pushed to the Wayback Machine, where they will persist even if Twitter adds authentication to the original URLs.
% cat run-me-ia.sh
#!/bin/csh -x
curl -L
"https://web.archive.org/web/20221208194342id_/https://video.twimg.com/dm_video/1600877027330064385/vid
/0/0/320x180/jZYOJeLERXPOC4qe.mp4" > ia-00.mp4
curl -L
"https://web.archive.org/web/20221208194342id_/https://video.twimg.com/dm_video/1600877027330064385/vid
/0/3000/320x180/l1mZtezfzjRRYziE.m4s " > ia-01.m4s
curl -L
"https://web.archive.org/web/20221208194342id_/https://video.twimg.com/dm_video/1600877027330064385/vid
@phonedude
phonedude / grab-twitter-video-curl.txt
Created December 12, 2022 22:50
Grabbing the video files from twitter.com without HTTP authentication.
% cat run-me.sh
#!/bin/csh -x
curl "https://video.twimg.com/dm_video/1600877027330064385/vid/0/0/320x180/jZYOJeLERXPOC4qe.mp4" >
00.mp4
curl "https://video.twimg.com/dm_video/1600877027330064385/vid/0/3000/320x180/l1mZtezfzjRRYziE.m4s " >
01.m4s
curl "https://video.twimg.com/dm_video/1600877027330064385/vid/3000/6000/320x180/yxhsDmzuJG9ZqtYb.m4s "
> 02.m4s
curl "https://video.twimg.com/dm_video/1600877027330064385/vid/6000/9000/320x180/KGtEzr2KaRfP4Y6H.m4s "
> 03.m4s
@phonedude
phonedude / curl-video-playlist.txt
Created December 12, 2022 22:34
curl on the Twitter DM video URL; returns a Media Playlist (RFC 8216)
% curl -i "https://video.twimg.com/dm_video/1600877027330064385/pl/320x180/Vn4h39llbQ0jfr1D.m3u8?container=fmp4"
HTTP/2 200
perf: 7626143928
content-type: application/x-mpegURL
cache-control: max-age=604800, must-revalidate
last-modified: Thu, 08 Dec 2022 15:34:40 GMT
x-transaction-id: 8d01a92de1228077
timing-allow-origin: https://twitter.com, https://mobile.twitter.com
x-content-type-options: nosniff
strict-transport-security: max-age=631138519
@phonedude
phonedude / twitter-dm-image-session-over-curl.txt
Last active December 12, 2022 20:41
"Copy as curl" from browser, but since the Twitter DM session is over, the server returns a 401 instead of a 200.
% curl -i 'https://ton.twitter.com/i/ton/data/dm/1600870219324465156/1600870190459256832/KM0EBzij.jpg:small' \
-X 'GET' \
-H 'Cookie: _ga=GA1.2.1578974351.1658522778; _gid=GA1.2.1415303809.1669565137; twid=u%3D1600710747004305409; att=1-n11EVoxa8oM767165HUbjPZ8YJFiYvFZMzw3uCFr; auth_token=88ba7e40f9067e
ec06c5501f86da52c9658dbf3b; ct0=43a7ed19a7f2dd7d0f8cf464bb9ac2b31fcacf0288a38c2c8a0773a2327a4f7d82b02030b5255758b42c5cd26ff817784a488a3fbd108dff3963f4ba66b3a1d46cd2ac8031028e57f6a3bd
25253aae40; kdt=1pjaQNrUHWOACrgtiPbMODtq9EtyfO2zjyqIRn2M; _twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCEj9I
POEAToMY3NyZl9p%250AZCIlZWQ5OGM3NWE3NjliNWM5YTExYTAyODEwZjEzMzE2ZmQ6B2lkIiUxOGI2%250AZjk4Y2Q2NTlmMjI1YWQxNDAxZTEyNjhhNjI5NA%253D%253D--8432d160885b961f50dd7ad4bfac669c10cd8f1b; gt=16
00929594542067725; dnt=1; guest_id=v1%3A167052633266899352; personalization_id="v1_I8bVcj9tTkUu6IElXKvpxA=="; guest_id_ads=v1%3A157695208658630181; guest_id_marketing=v1
@phonedude
phonedude / twitter-dm-image-curl.txt
Last active December 13, 2022 05:04
"Copy as curl" from browser, we can save an image from a Twitter DM.
% curl 'https://ton.twitter.com/i/ton/data/dm/1600870219324465156/1600870190459256832/KM0EBzij.jpg:small' \
-X 'GET' \
-H 'Cookie: _ga=GA1.2.1578974351.1658522778; _gid=GA1.2.1415303809.1669565137; twid=u%3D1600710747004305409; att=1-EAfZonmiIFAabN7WJFAYruyozkoR4R1UXTi9Mrbd; auth_token=0f57cc642e5efc86e77113c4c6e26e63f4a8277c; ct0=6affe5131db8f27b7e737d10cc4db10ba7a18e09c64ae1049daf2113619feb98bb3fc2d2c2a9dc0bbb4890329e2871639a67cdd121c5962552fdae23dc2238c776da3a6296484e3b51ee9da53abb120a; kdt=1pjaQNrUHWOACrgtiPbMODtq9EtyfO2zjyqIRn2M; _twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCOIYHPCEAToMY3NyZl9p%250AZCIlNWZjNjY2YTUwMjIwYjY2ODU0OGMzNDFlNjdmYzg2NmU6B2lkIiVkZTg1%250ANjg4ZjJkMDZjZGM1MDlhNjIyMjJlMDQ5ZTA2NQ%253D%253D--5208257722568ae7fb82098c4a047426afb617b2; external_referer=5R4llnU6LiS6nKIFlrLdVgSXOiDO0qILODRlwl3mUPKfPbBdozq%2F%2BhlJutu4u37W|1|8e8t2xd8A2w%3D; dnt=1; guest_id=v1%3A167047567821699925; personalization_id="v1_jeT69pPEhhQw02