rain-1

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

moyix

Using Linear Algebra to Convert a Large Code Model

Background

The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

• nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
• multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
• mono, which is fine-tuned from the multi model on Python code only.

aparrish

JimRoepcke

// Playground - noun: a place where people can play

import Cocoa

struct Regex {
    let pattern: String
    let expressionOptions: NSRegularExpressionOptions
    let matchingOptions: NSMatchingOptions
    
    init(pattern: String, expressionOptions: NSRegularExpressionOptions, matchingOptions: NSMatchingOptions) {

Trie.swift

//
//  Trie.swift
//  SceneTest
//
//  Created by Greg Titus on 6/7/14.
//  Copyright (c) 2014 The Omni Group. All rights reserved.
//

// This is essentially the same data structure as OFTrie in OmniFoundation, except that the OmniFoundation version is mutable and ephemeral,
// and this version is immutable and persistent. Every time you add a string to the trie, it replaces all the nodes along the string's path