Skip to content

Instantly share code, notes, and snippets.

View Laeeth's full-sized avatar

Laeeth Isharc Laeeth

View GitHub Profile
@MIvanchev
MIvanchev / article.md
Last active April 4, 2023 13:39
Ever wondered what it takes to run Windows software on ARM? Then this article might be for you!
@saharNooby
saharNooby / RWKV_cache.py
Last active May 3, 2023 20:58
State cache for RWKV language model
# USAGE EXAMPLE
cache = RWKV_Cache()
init_out, init_state = cache.preprocess_prompt(model, prompt_tokens)
for GENERATION in range(NUM_GENERATIONS):
out, state = init_out.clone(), init_state.clone()
cache_key = [*prompt_tokens]
@saharNooby
saharNooby / simple_bpe_tokenizer.py
Last active May 5, 2023 18:57
Probably the dumbest, no-dependencies, pure Python implementation of 20B_tokenizer.json (a BPE tokenizer for GPT-NeoX model)
import regex
import json
import unicodedata
from typing import Tuple, Callable, Union
# Parses the tokenizer config and returns encode and decode functions.
def load_tokenizer(config_path: str) -> Tuple[Callable[[str], list[int]], Callable[[list[int]], str]]:
# Maps any byte 0..255 to a printable Unicode character.
byte_to_unicode: dict[int, str] = {
33: "!",
@elliottmorris
elliottmorris / historical_approval.R
Last active July 29, 2023 14:00
historical presidential approval ratings
rm(list = ls()) #reset the environment
library(tidyverse)
library(lubridate)
library(mgcv)
exponent_weight <- function(i) {
exp(-0.04*i)
}
TODAY_DAY <- difftime(Sys.Date(),as.Date("2017-01-21"))
@Laeeth
Laeeth / long_gpt.py
Created April 13, 2023 04:17 — forked from NaxAlpha/long_gpt.py
Training script for LongGPT; Fine-tunes GPT-2 (335M) on The Pile Dataset with a context size of 8k tokens. (requires > 16GB RAM)
import time
from contextlib import suppress
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cuda as cuda
from torch.utils.data import DataLoader, IterableDataset
Type your search: What is Germany?
(Document(page_content="Germany has been described as a [[great power]] with [[Economy of Germany|a strong economy]]; it has the [[List of sovereign states in Europe by GDP (nominal)|largest economy in Europe]], the world's [[List of countries by GDP (nominal)|fourth-largest economy by nominal GDP]] and the [[List of countries by GDP (PPP)|fifth-largest by PPP]]. As a global power in industrial, [[Science and technology in Germany|scientific and technological]] sectors, it is both the world's [[List of countries by exports|third-largest exporter]] and [[List of countries by imports|importer]]. As a [[developed country]] it [[Social security in Germany|offers social security]], [[Healthcare in Germany|a universal health care system]] and [[Higher education in Germany|a tuition-free university education]]. Germany is a member of the [[United Nations]], the European Union, [[NATO]], the [[Council of Europe]], the [[G7]], the [[G20]] and the [[OECD]]. It has the [[List of World
@NaxAlpha
NaxAlpha / long_gpt.py
Last active October 15, 2023 11:21
Training script for LongGPT; Fine-tunes GPT-2 (335M) on The Pile Dataset with a context size of 8k tokens. (requires > 16GB RAM)
import time
from contextlib import suppress
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cuda as cuda
from torch.utils.data import DataLoader, IterableDataset
@jiamingkong
jiamingkong / langchain.ipynb
Created May 1, 2023 04:07
Using RWKV in langchain
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@stain
stain / .notmuch-config
Last active November 13, 2023 08:17
NotMuch hooks for selectively sorting to inbox, moving archived messages to other folder. offlineimap syncs to /home/USER/mail/company and /home/USER/mail/org1
[database]
path=/home/USER/mail
[user]
name=MyName MySurname
primary_email=me@example.com
other_email=me@example.org
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.