Skip to content

Instantly share code, notes, and snippets.

View Laeeth's full-sized avatar

Laeeth Isharc Laeeth

View GitHub Profile
@jiamingkong
jiamingkong / langchain.ipynb
Created May 1, 2023 04:07
Using RWKV in langchain
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@NaxAlpha
NaxAlpha / long_gpt.py
Last active October 15, 2023 11:21
Training script for LongGPT; Fine-tunes GPT-2 (335M) on The Pile Dataset with a context size of 8k tokens. (requires > 16GB RAM)
import time
from contextlib import suppress
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cuda as cuda
from torch.utils.data import DataLoader, IterableDataset
Type your search: What is Germany?
(Document(page_content="Germany has been described as a [[great power]] with [[Economy of Germany|a strong economy]]; it has the [[List of sovereign states in Europe by GDP (nominal)|largest economy in Europe]], the world's [[List of countries by GDP (nominal)|fourth-largest economy by nominal GDP]] and the [[List of countries by GDP (PPP)|fifth-largest by PPP]]. As a global power in industrial, [[Science and technology in Germany|scientific and technological]] sectors, it is both the world's [[List of countries by exports|third-largest exporter]] and [[List of countries by imports|importer]]. As a [[developed country]] it [[Social security in Germany|offers social security]], [[Healthcare in Germany|a universal health care system]] and [[Higher education in Germany|a tuition-free university education]]. Germany is a member of the [[United Nations]], the European Union, [[NATO]], the [[Council of Europe]], the [[G7]], the [[G20]] and the [[OECD]]. It has the [[List of World
@Laeeth
Laeeth / long_gpt.py
Created April 13, 2023 04:17 — forked from NaxAlpha/long_gpt.py
Training script for LongGPT; Fine-tunes GPT-2 (335M) on The Pile Dataset with a context size of 8k tokens. (requires > 16GB RAM)
import time
from contextlib import suppress
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cuda as cuda
from torch.utils.data import DataLoader, IterableDataset
@elliottmorris
elliottmorris / historical_approval.R
Last active July 29, 2023 14:00
historical presidential approval ratings
rm(list = ls()) #reset the environment
library(tidyverse)
library(lubridate)
library(mgcv)
exponent_weight <- function(i) {
exp(-0.04*i)
}
TODAY_DAY <- difftime(Sys.Date(),as.Date("2017-01-21"))
@saharNooby
saharNooby / simple_bpe_tokenizer.py
Last active May 5, 2023 18:57
Probably the dumbest, no-dependencies, pure Python implementation of 20B_tokenizer.json (a BPE tokenizer for GPT-NeoX model)
import regex
import json
import unicodedata
from typing import Tuple, Callable, Union
# Parses the tokenizer config and returns encode and decode functions.
def load_tokenizer(config_path: str) -> Tuple[Callable[[str], list[int]], Callable[[list[int]], str]]:
# Maps any byte 0..255 to a printable Unicode character.
byte_to_unicode: dict[int, str] = {
33: "!",
@saharNooby
saharNooby / RWKV_cache.py
Last active May 3, 2023 20:58
State cache for RWKV language model
# USAGE EXAMPLE
cache = RWKV_Cache()
init_out, init_state = cache.preprocess_prompt(model, prompt_tokens)
for GENERATION in range(NUM_GENERATIONS):
out, state = init_out.clone(), init_state.clone()
cache_key = [*prompt_tokens]
@MIvanchev
MIvanchev / article.md
Last active April 4, 2023 13:39
Ever wondered what it takes to run Windows software on ARM? Then this article might be for you!
@rolandcrosby
rolandcrosby / gist:c26571bf4e263f695d2f
Last active March 27, 2023 18:34
Convert rich text on the clipboard to Markdown
if encoded=`osascript -e 'the clipboard as «class HTML»'` 2>/dev/null; then echo $encoded | perl -ne 'print chr foreach unpack("C*",pack("H*",substr($_,11,-3)))' | pandoc --no-wrap -f HTML -t markdown; else; pbpaste; fi
# for my .vimrc:
# command PasteMarkdown :read !if encoded=`osascript -e 'the clipboard as «class HTML»'` 2>/dev/null; then echo $encoded | perl -ne 'print chr foreach unpack("C*",pack("H*",substr($_,11,-3)))' | pandoc --no-wrap -f HTML -t markdown; else; pbpaste; fi
# nnoremap ,pmd :PasteMarkdown<CR>
@Khady
Khady / config.el
Last active February 27, 2023 12:25
OCaml and Reasonml emacs configuration
(use-package company
:ensure t
:custom
(company-quickhelp-delay 0)
(company-tooltip-align-annotations t)
:hook
((prog-mode utop-mode) . company-mode)
:config
(company-quickhelp-mode 1)
:bind