This was a response to a Hacker News comment asking me what I've been up to since 2010. I'm posting it here since HN rejects it with "that comment is too long." I suppose that's fair, since this ended up being something of an autobiography.
--
What happened after 2010?
The Verge: "Meta’s powerful AI language model has leaked online — what happens now?"
Could you confirm that you downloaded the LLaMA series from 4chan? Were you able to get it running yourself or did you just repackage the download? (I was a bit confused reading your tweets about that what exactly you'd done there, so if you're able to explain that, it'd be great)
I downloaded it from Facebook, actually. You can find some details here.
Basically, the sequence of events was:
| ./tokenizer_checklist.chk 50 | |
| ./tokenizer.model 499723 | |
| ./7B/checklist.chk 100 | |
| ./7B/consolidated.00.pth 13476939516 | |
| ./7B/params.json 101 | |
| ./13B/checklist.chk 154 | |
| ./13B/consolidated.00.pth 13016334699 | |
| ./13B/consolidated.01.pth 13016334699 | |
| ./13B/params.json 101 | |
| ./30B/checklist.chk 262 |
JAX released a persistent compilation cache for TPU VMs! When enabled, the cache writes compiled JAX computations to disk so they don’t have to be re-compiled the next time you start your JAX program. This can save startup time if any of y’all have long compilation times.
First upgrade to the latest jax release:
pip install -U "jax[tpu]>=0.2.18" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html
Then use the following to enable the cache in your jax code:
from jax.experimental.compilation_cache import compilation_cache as cc| import os | |
| import sys | |
| import re | |
| import time | |
| import psutil | |
| import platform | |
| import json | |
| import collections | |
| from subprocess import check_output |
| #include <glob.h> | |
| #include <vector> | |
| #include <string> | |
| namespace util | |
| { | |
| std::vector<std::string> glob(const std::string& pattern) { | |
| glob_t glob_result = {0}; // zero initialize |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| {"seed": 374894, "temp": 0.7, "top_p": 0.0, "top_k": 40, "repetition_penalty": 1.1764705882352942, "max_seq_len": 512, "max_gen_len": 511} | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| Loading | |
| Loaded in 8.72 seconds | |
| ============== sample 1 ================= | |
| I believe the meaning of life is to grow, learn and give. |
| cmake_policy(VERSION "3.25.0") | |
| set(reserved | |
| ALL | |
| "=" ON | |
| "==" ON | |
| "+" ON | |
| "_" ON | |
| "%" ON | |
| "*" ON | |
| "/" ON |
Twitter thread: https://twitter.com/theshawwn/status/1456925974919004165
Hacker News thread: https://news.ycombinator.com/item?id=29128998
November 6, 2021
jnp.device_put(1) is deceptively simple to write in JAX. But on a TPU, what actually happens? How does a tensor containing the value 1 actually get onto a TPU?
Turns out, the answer is "C++", and a lot of it.
| You have fallen into Event Horizon with John Michael Gadia. | |
| In today's episode, John is joined by Sean Pracer. | |
| Sean Pracer is an AI researcher and machine learning engineer. | |
| He has contributed to projects such as ThePile, an open source | |
| training data set for large language models. | |
| He currently works on research and development for AGI. |