Pre-reqs:
| <? | |
| ///////////////////// | |
| // slack2html | |
| // by @levelsio | |
| ///////////////////// | |
| // | |
| ///////////////////// | |
| // WHAT DOES THIS DO? | |
| ///////////////////// | |
| // |
| /* | |
| the twitter api is stupid. it is stupid and bad and expensive. hence, this. | |
| Literally just paste this in the JS console on the bookmarks tab and the script will automatically scroll to the bottom of your bookmarks and keep a track of them as it goes. | |
| When finished, it downloads a JSON file containing the raw text content of every bookmark. | |
| for now it stores just the text inside the tweet itself, but if you're reading this why don't you go ahead and try to also store other information (author, tweetLink, pictures, everything). come on. do it. please? | |
| */ |
| """ | |
| a simple script that reads tweets inside a json file, uses openai to compute embeddings and creates two files, metadata.tsv and output.tsv, which cam be used to visualise the tweets and their embeddings in TensorFlow Projector (https://projector.tensorflow.org/) | |
| """ | |
| # obtain tweets.json from https://gist.github.com/gd3kr/948296cf675469f5028911f8eb276dbc | |
| import pandas as pd | |
| import json | |
| from openai import OpenAI |
| //there are issues with how we manage to skip pixels. | |
| //accounts are getting skipped. | |
| let following = []; // Initialize an empty array to hold all elements of people you follow | |
| let formattedFollowers = [] | |
| const scrollInterval = 2000; | |
| const scrollStep = 2000; // Pixels to scroll on each step | |
| let previousFollowingCount = 0; |
| /* Enhancements to the Twitter Scraping Script: | |
| This update to the script introduces a more robust mechanism for extracting detailed interaction data from tweets as they are scraped from Twitter. Previously, the script focused on collecting basic content such as the tweet's text. Now, it has been augmented to include a comprehensive extraction of interaction metrics, including replies, reposts, likes, bookmarks, and views, for each tweet. | |
| Key Changes: | |
| 1. Improved Data Extraction: | |
| - The script now searches through all elements within a tweet that have an `aria-label` attribute, filtering for labels that contain key interaction terms (replies, reposts, likes, bookmarks, views). This ensures that only relevant `aria-labels` are considered for data extraction. | |
| 2. Flexible Interaction Data Parsing: |
| import cv2 | |
| # Windows dependencies | |
| # - Python 2.7.6: http://www.python.org/download/ | |
| # - OpenCV: http://opencv.org/ | |
| # - Numpy -- get numpy from here because the official builds don't support x64: | |
| # http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy | |
| # Mac Dependencies | |
| # - brew install python |
| <? | |
| ///////////////////// | |
| // slack2html | |
| // by @levelsio | |
| ///////////////////// | |
| // | |
| ///////////////////// | |
| // WHAT DOES THIS DO? | |
| ///////////////////// | |
| // |
| // | |
| // ContentView.swift | |
| // coremlsd2test | |
| // | |
| // Created by Yasuhito Nagatomo on 2022/12/03. | |
| // | |
| // A sample code using Apple/ml-stable-diffusion library. | |
| // Preparation: | |
| // 1. convert the PyTorch Stable-Diffusion v2 model to coreml models using Apple's tools. | |
| // 2. import the coreml models into the iOS project. |