veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

scateu

# Open "Automator.app", add a new "Quick Action"

# Workflow receives current "files or folders" in "Finder"
#+ Run Shell Script
#+ Shell: /bin/bash
#+ Pass input: as arguments

# Deployment: Keyboard / Shortcuts, bind 'Cmd-Shift-V'
# Usage: copy something to clipboard, click on a folder, press 'Cmd-Shift-V'. 
#        Contents will be pasted into 'pbpaste.txt'. 

apaszke

import torch                                                                                          
                                                                                                      
def jacobian(y, x, create_graph=False):                                                               
    jac = []                                                                                          
    flat_y = y.reshape(-1)                                                                            
    grad_y = torch.zeros_like(flat_y)                                                                 
    for i in range(len(flat_y)):                                                                      
        grad_y[i] = 1.                                                                                
        grad_x, = torch.autograd.grad(flat_y, x, grad_y, retain_graph=True, create_graph=create_graph)
        jac.append(grad_x.reshape(x.shape))                                                      

yzh119

from __future__ import print_function
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable

def sample_gumbel(shape, eps=1e-20):
    U = torch.rand(shape).cuda()
    return -Variable(torch.log(-torch.log(U + eps) + eps))

kmatt

./configure --with-features=huge --enable-pythoninterp --with-python-config-dir=~/anaconda/bin/python-config

lsd

Upgrading to Lion or Yosemite and WebStorm 9, I noticed key repeat was
turned off for the IdeaVim plugin h j k l keys.

System-wide key repeat

defaults write -g ApplePressAndHoldEnabled -bool false in a terminal will enable
key repeat for every app. This can alternatively be found in the accessibility settings in OS X' preferences.

App specific key repeat