Skip to content

Instantly share code, notes, and snippets.

@kinoc
kinoc / j6b_train_hf_ds.py
Last active August 9, 2023 03:05
So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay, using HF and DeepSpeed too
# So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay
# More exploratory coding. It uses the Huggingface model port, deepspeed and reads all text/md files from a target directory
# It is a fragment of a larger system with remote editing, but that's another story
# This is the raw, training tester. Items to look out for:
# - uses DeepSpeed and has a DS config
# - to save space uses SGD instead of ADAM
# - uses gradient checkpointing
# - freezes 25% of the layers to fit
# Assumes you can already run https://gist.github.com/kinoc/2d636a68876cd3de7b6e9c9452b61089
#!/usr/bin/env python3
import time
import os
import sys
import requests
import colorama
import queue
import statistics
from termcolor import colored
@turbo
turbo / c.md
Last active September 17, 2020 23:30

The Comic

This is an irregularly updated comic about things. It might contain inside jokes. The strips all have alt text.

#2 Job Golfing

#1 Adventures of Uberman

@kvz
kvz / Makefile
Last active August 28, 2021 14:35
The only Makefile for Node.js projects you'll ever need - https://twitter.com/kvz/status/685853830425231361
# Licensed under MIT.
# Copyright (2016) by Kevin van Zonneveld https://twitter.com/kvz
#
# This Makefile offers convience shortcuts into any Node.js project that utilizes npm scripts.
# It functions as a wrapper around the actual listed in `package.json`
# So instead of typing:
#
# $ npm script build:assets
#
# you could just as well type:
@squarism
squarism / iterm2.md
Last active April 25, 2024 03:50
An iTerm2 Cheatsheet

Tabs and Windows

Function Shortcut
New Tab + T
Close Tab or Window + W (same as many mac apps)
Go to Tab + Number Key (ie: ⌘2 is 2nd tab)
Go to Split Pane by Direction + Option + Arrow Key
Cycle iTerm Windows + backtick (true of all mac apps and works with desktops/mission control)
@ayshen
ayshen / everything-is-beautiful-and-nobody-is-happy.md
Last active September 16, 2019 02:44
Everything's beautiful, and nobody's happy. Why?

Everything's beautiful, and nobody's happy

There's no shortage of good-looking apps these days. But spend five minutes on Hacker News or reddit and you'll probably find at least one of these things:

  • A new app that puts a fresh coat of paint on something that had been completely ordinary until the developers got their grubby paws on it.
  • A short, pointed complaint about how some app or other "sucks" or is "unusable."
  • A redesign of said app.
  • A self-righteous blog post about user experience patterns.

A lot of the things that are going around are actually quite aesthetically pleasing. iOS, Android, Windows Phone, and the web have all come a long way since the era of Under Construction GIFs and webrings, and have each refined their graphic languages and interaction guidelines to some semblance of an "intuitive interface."

@joshskeen
joshskeen / android state codes string array
Created November 21, 2013 18:15
android state codes string array
<string-array name="states">
<item>AL</item>
<item>AR</item>
<item>AZ</item>
<item>CA</item>
<item>CO</item>
<item>CT</item>
<item>DE</item>
<item>DC</item>
<item>FL</item>