kinoc

#  So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay 
#  More exploratory coding. It uses the Huggingface model port, deepspeed and reads all text/md files from a target directory
#  It is a fragment of a larger system with remote editing, but that's another story
#  This is the raw, training tester. Items to look out for:
#  - uses DeepSpeed and has a DS config
#  - to save space uses SGD instead of ADAM
#  - uses gradient checkpointing
#  - freezes 25% of the layers to fit

# Assumes you can already run https://gist.github.com/kinoc/2d636a68876cd3de7b6e9c9452b61089

travisbrown

Recent tweets by Alexy Khrabrov, organizer of the SF Scala meetup and Scale by the Bay (a Scala conference).

Update: Khrabrov has written a couple of responses and deleted both. In the second he writes:

Folks -- these tweets are taken out of context, mistranslated, and are used as a cudgel on Twitter to trigger others into canceling communities and people. They do not represent my views on race in any way, for instance.

The links below are to archives that provide the full context, and all of these quotations are verbatim (I had originally copied the Twitter-provided English translation of one Russian phrase, but have removed that for clarity).

Update: The Wayback Machine links below currently

001101

CREATE TABLE accounts(
	id serial PRIMARY KEY,
	name VARCHAR(256) NOT NULL
);

CREATE TABLE entries(
	id serial PRIMARY KEY,
	description VARCHAR(1024) NOT NULL,
	amount NUMERIC(20, 2) NOT NULL CHECK (amount > 0.0),
	-- Every entry is a credit to one account...

olih

Processing JSON using jq

jq is useful to slice, filter, map and transform structured json data.

Installing jq

On Mac OS

brew install jq