veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

wklchris

Main steps

Total 4 steps for connect & exit Jupyter Notebook with a SSH server:

1. SSH to the Server. Run:

   jupyter notebook --no-browser --port=8888

2. Open another terminal window on your local machine, input:

crazymuse

# Markov Decision Process Example
import numpy as np
import copy

V ={"dragonstone": 0,"whiteharbor":0, "winterfell":0,"alive-terminal":1,"dead-terminal":-1} # States

R ={"from_dragonstone":{"land":-0.02,"sea":-0.05,"dragon":-0.1},\
    "from_whiteharbor":{"land":-0.01},\
   "from_winterfell":{"land":-0.01},\
   }

RatulSaha

#List traversal
range(start, stop, hop)
range(n) # [0,1,...,n-1]
range(1,n) # [1,...,n-1]
range(1,n,2) # [1,3,5,...,n-1] if n is even, or [1,3,5,...,n-2] if n is odd
range(n,-1,-1) # [n,n-1,n-2,...,0]
range(len(arr)) # Provides indices of an array arr
range(len(arr)-1,-1,-1) # Provides indices of arr backwards

# List slicing

karpathy

""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
import numpy as np
import cPickle as pickle
import gym

# hyperparameters
H = 200 # number of hidden layer neurons
batch_size = 10 # every how many episodes to do a param update?
learning_rate = 1e-4
gamma = 0.99 # discount factor for reward

dannguyen

Using Python 3 + Google Cloud Vision API's OCR to extract text from photos and scanned documents

Just a quickie test in Python 3 (using Requests) to see if Google Cloud Vision can be used to effectively OCR a scanned data table and preserve its structure, in the way that products such as ABBYY FineReader can OCR an image and provide Excel-ready output.

The short answer: No. While Cloud Vision provides bounding polygon coordinates in its output, it doesn't provide it at the word or region level, which would be needed to then calculate the data delimiters.

On the other hand, the OCR quality is pretty good, if you just need to identify text anywhere in an image, without regards to its physical coordinates. I've included two examples:

####### 1. A low-resolution photo of road signs

williscool

// permutations(["c","a","t"])
function permutations(array){
  
  if (array.length === 0) return [[]];
  
  var perms = [];
  
  for(var i = 0; i < array.length; i++) {
    
    var copy = array.slice(0);

vasanthk

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

adrianh

Copy/paste from my read.md file ;)

Jan 2015

• A Slip of the Keyboard, Terry Pratchett - Mildly entertaining, if a bit repetitive in places. Especially since it was grouped into themes.
• Ax, Ed McBain - Reminding myself how good a writer he is.
• Replay, Ken Grimwood - Somehow avoided reading this until now. Fun.
• Master & Commander, Patrick O'Brian - Starting my Aubrey–Maturin re-re-read
• Post Captain, Patrick O'Brian - Continuing my Aubrey–Maturin re-re-read
• The Peripheral, William Gibson — Didn't enjoy this as much as the Red Ant books. But less than excellent Gibson is still head and shoulders above most writers.

karpathy

"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np

# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)