This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# A port of https://github.com/karpathy/llama2.c/blob/master/run.c | |
# to Julia. | |
# Jiahao Chen <jiahao@csail.mit.edu> 2023-07-29 | |
# | |
# MIT License: see full text at https://opensource.org/license/mit/ | |
# | |
using LinearAlgebra | |
using LogExpFunctions |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using Plots | |
using StatsPlots | |
using LinearAlgebra | |
using ClassicalOrthogonalPolynomials | |
using ProgressMeter | |
using Statistics | |
k = 15 # Size of training data | |
l = 15 # Size of test data |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using Statistics | |
using BFloat16s | |
using StaticArrays | |
import Base: getindex, setindex!, length, iterate | |
########################################### | |
# Implementation of the NormedFloat4 type | |
# and its container type, QLoRAArray | |
# |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from bs4 import BeautifulSoup | |
import urllib.request | |
url = "https://arxiv.org/a/chen_j_2.html" | |
with urllib.request.urlopen(url) as response: | |
html = response.read() | |
soup = BeautifulSoup(html, 'html.parser') | |
for link in soup.find_all('a'): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using Dates: now | |
using DataFrames | |
import Base: *, push! | |
mutable struct AAUpdate{Tu,Tv} # Matrix-free representation of the H matrix | |
m::Int #:: Size of the AA subspace | |
u::Vector{Tu} #:: The quantities s-Hỹ (note typo in paper) | |
v::Vector{Tv} #:: The quantities (H'ŝ)/(ŝ'Hŷ) | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Iterate over SparseMatrixCSC stored entries, ignoring stored zeros and | |
# missing values. | |
# | |
# Implements Julia's new iterator protocol (new as of v0.7) | |
# Ref: https://julialang.org/blog/2018/07/iterators-in-julia-0.7 | |
# | |
# Jiahao Chen 2019-12-03 | |
# | |
# MIT License available upon request | |
# |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using LinearAlgebra | |
using StatsBase | |
using StatsFuns | |
using NaNMath | |
"x->2x-1 in place" | |
function twoxm1!(dat; val=0.0) | |
@inbounds for (i,x) in enumerate(dat) | |
dat[i] = ifelse(isnan(x), val, 2x-1) | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Keybase proof | |
I hereby claim: | |
* I am jiahao on github. | |
* I am jiahao (https://keybase.io/jiahao) on keybase. | |
* I have a public key ASD8uXwFCTxC_HNYH0M6m_5niip3vql6gQ9nqYuUWnkiiQo | |
To claim this, I am signing this object: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
struct MultinomialNaiveBayes{T, V<:AbstractVector} | |
feature_ratios::V | |
prior_ratio::T | |
end | |
""" | |
fit(MultinomialNaiveBayes, [T,] features, labels, α = 1) -> MNB | |
fits a `MultinomialNaiveBayes` classifier `MNB` using the | |
`features` matrix and `labels` vector of `Bool`s. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
struct NaiveBayes{T, V<:AbstractVector, M<:AbstractMatrix} | |
probabilities::M | |
priors::V | |
end | |
train(::Type{NaiveBayes}, T::Type{R}, features, labels, α = 1) where R<:Real = | |
train(NaiveBayes{T, Vector{T}, Matrix{T}}, features, labels, α) | |
for (typ, op) in ((Rational, ://), (Real, :/)) @eval begin | |
function train(::Type{NaiveBayes{T, S, R}}, |
NewerOlder