theodesp

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts and experience preferred (super rare at this point).

My own notes from a few months back.

Background

• Survey of LLMS
• Self-attention and transformer networks

theodesp

Sam Altman: "I think if you have a smart person who has learned to do good research and has the right sort of mindset, it only takes about six months to make them, you know, take a smart physics researcher and make them into a productive AI researcher. So we don't have enough talent in the field yet, but it's coming soon. We have a program at open AI that does exactly this. And I'm astonished how well it works."

Types of roles at AI companies

• Software Engineer (not the focus of this Gist): Build customer-facing features, optimize applications for speed and scale, use AI APIs. Prompt engineering expertise is generally helpful, but AI experience beyond using the APIs or using ChatGPT like an expert is generally not needed. This Gist isn't aimed at this role.
• Machine Learning Engineer: Build pipelines for data management, model training, and model deployment, to improve models (not the focus of this Gist). And/or implement cutting-edge research papers (a focus of this Gist).
• Research Engineer (a *

theodesp

const std = @import("std");
const builtin = @import("builtin");
const assert = std.debug.assert;
const Allocator = std.mem.Allocator;
const ArenaAllocator = std.heap.ArenaAllocator;
const StreamingParser = std.json.StreamingParser;
const Token = std.json.Token;
const TokenTag = std.meta.FieldEnum(Token);

/// Field options are options that can be set per-field on a struct at

theodesp

#pragma once

#define DYN_ARR_OF(type) struct  { \
  type *data; \
  type *endptr; \
  uint32_t capacity; \
}

#if !defined(__cplusplus)
#define decltype(x) void*

theodesp

Recon and Attack Vectors from My Logs

This document contains excerpts from my web server logs collected over a period of 7 years that shows various kinds of recon and attack vectors.

There were a total of 37.2 million lines of logs out of which 1.1 million unique HTTP requests (Method + URI) were found.

$ sed 's/^.* - - \[.*\] "\(.*\) HTTP\/.*" .*/\1/' access.log > requests.txt

theodesp

// This is a proper alternative to
// https://github.com/BuckleScript/bucklescript/blob/b9508105b1a35537bdea9a1fabd10f6c65f776b4/jscomp/bsb/templates/react-hooks/src/FetchedDogPictures/FetchedDogPictures.re#L14
// The one in that file uses Promise, but that's *wrong*.
// We only used promise as a demo of its API. We'll remove it soon.

// As you can see below, the pure XMLHttpRequest code is just as clean,
// less mysterious for all, more performant, extensible, and actually correct.

// Ignore these externals for now. They're just for illustration
// purposes. I just copy pasted the Js code from