veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

veekaybee

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

prologic

Learn Go in ~5mins

This is inspired by A half-hour to learn Rust and Zig in 30 minutes.

Basics

Your first Go program as a classical "Hello World" is pretty simple:

First we create a workspace for our project:

wrobstory

fredbenenson

layout	title	description	tags
default	SQL Style Guide	A guide to writing clean, clear, and consistent SQL.	data process

Purpose

iconara

-- Table information like sortkeys, unsorted percentage
-- see http://docs.aws.amazon.com/redshift/latest/dg/r_SVV_TABLE_INFO.html
SELECT * FROM svv_table_info;

-- Table sizes in GB
SELECT t.name, COUNT(tbl) / 1000.0 AS gb
FROM (
  SELECT DISTINCT datname, id, name
  FROM stv_tbl_perm
  JOIN pg_database ON pg_database.oid = db_id

jamtur01

These are the Kickstarter Engineering and Data role definitions for both teams.

Technical	Data	People
Junior Software Engineer
Software Engineer	Data Analyst
Senior Software Engineer	Data Scientist	Engineering Manager
Staff Engineer	VP of Data	Engineering Director
Principal Engineer		CTO

phobson

wrobstory

-- Gets all queries for a given date range
select starttime, endtime, trim(querytxt) as query
from stl_query
where starttime between '2014-11-04' and '2014-11-05'
order by starttime desc;

-- Gets all queries that have been aborted for a given date range
select starttime, endtime, trim(querytxt) as query, aborted
from stl_query 
where aborted=1

treycausey

# Slopegraphs in matplotlib
# Trey Causey (@treycausey)
# Problems arise when equal values occur
# within the same time unit

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

units = ['A', 'B', 'C', 'D']