Skip to content

Instantly share code, notes, and snippets.

Avatar
⚒️
🦀 🐿️ 🐘 🐳⚛️ 🚢 🚀 🦄 🍵

Cedric Chee cedrickchee

⚒️
🦀 🐿️ 🐘 🐳⚛️ 🚢 🚀 🦄 🍵
View GitHub Profile
@cedrickchee
cedrickchee / ai-plugin.json
Created March 25, 2023 14:22 — forked from danielgross/ai-plugin.json
ChatGPT Plugin for Twilio
View ai-plugin.json
{
"schema_version": "v1",
"name_for_model": "twilio",
"name_for_human": "Twilio Plugin",
"description_for_model": "Plugin for integrating the Twilio API to send SMS messages and make phone calls. Use it whenever a user wants to send a text message or make a call using their Twilio account.",
"description_for_human": "Send text messages and make phone calls with Twilio.",
"auth": {
"type": "user_http",
"authorization_type": "basic"
},
@cedrickchee
cedrickchee / alpaca-native-langchain-chatbot-tutorial.md
Last active March 28, 2023 23:47
Creating a chatbot using Alpaca native and LangChain
View alpaca-native-langchain-chatbot-tutorial.md

Creating a chatbot using Alpaca native and LangChain

Let's talk to an Alpaca-7B model using LangChain with a conversational chain and a memory window.

Setup and installation

Install python packages using pip. Note that you need to install HuggingFace Transformers from source (GitHub) currently.

$ pip install git+https://github.com/huggingface/transformers
@cedrickchee
cedrickchee / alpaca-inference.py
Last active March 22, 2023 19:34
HuggingFace Transformers inference for Stanford Alpaca (fine-tuned LLaMA)
View alpaca-inference.py
# Based on: Original Alpaca Model/Dataset/Inference Code by Tatsu-lab
import time, torch
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
tokenizer = LlamaTokenizer.from_pretrained("./checkpoint-1200/")
def generate_prompt(instruction, input=None):
if input:
return f"""The following is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
@cedrickchee
cedrickchee / llama-7b-m1.md
Last active May 28, 2023 21:10
4 Steps in Running LLaMA-7B on a M1 MacBook with `llama.cpp`
View llama-7b-m1.md

4 Steps in Running LLaMA-7B on a M1 MacBook

The large language models usability

The problem with large language models is that you can’t run these locally on your laptop. Thanks to Georgi Gerganov and his llama.cpp project, it is now possible to run Meta’s LLaMA on a single computer without a dedicated GPU.

Running LLaMA

There are multiple steps involved in running LLaMA locally on a M1 Mac after downloading the model weights.

@cedrickchee
cedrickchee / text-generation-webui-guide.md
Created March 12, 2023 12:04
Installing 8/4-bit LLaMA with text-generation-webui on Linux
View text-generation-webui-guide.md

Installing 8-bit LLaMA with text-generation-webui

Linux

  1. Follow the instructions here under "Installation option 1: conda"

  2. Download the desired Hugging Face converted model for LLaMA here

  3. Copy the entire model folder, for example llama-13b-hf, into text-generation-webuimodels

@cedrickchee
cedrickchee / meta-llama-guide.md
Created March 12, 2023 11:37
Meta's LLaMA 4-bit chatbot guide for language model hackers and engineer
View meta-llama-guide.md

info 9-3-23 Added 4bit LLaMA install instructions for cards as small as 6GB VRAM! (See "BONUS 4" at the bottom of the guide)

warning 9-3-23 Added Torrent for HFv2 Model Weights, required for ooga's webUI, Kobold, Tavern and 4bit (+4bit model)! Update ASAP!

danger 11-3-23 There's a new torrent version of the 4bit weights called "LLaMA-HFv2-4bit". The old "LLaMA-4bit" torrent may be fine. But if you have any issues with it, it's recommended to update to the new 4bit torrent or use the decapoda-research versions off of HuggingFace or produce your own 4bit weights. Newer Torrent Link or [Newer Magnet Link](magnet:?xt=urn:btih:36945b5958b907b3ab69e963ba0de1abdf48c16c&dn=LLaMA-HFv2-4bit&tr=http%3a%2f%2fbt1.archive.org%3a6969%2fannounce&tr=http%3a%2f%2fbt2.archive.org%3a696

@cedrickchee
cedrickchee / example.sh
Created March 6, 2023 13:28 — forked from shawwn/example.sh
How I run 65B using my fork of llama at https://github.com/shawwn/llama
View example.sh
mp=1; size=7B; # to run 7B
mp=8; size=65B; # to run 65B
for seed in $(randint 1000000)
do
export TARGET_FOLDER=~/ml/data/llama/LLaMA
time python3 -m torch.distributed.run --nproc_per_node $mp example.py --ckpt_dir $TARGET_FOLDER/$size --tokenizer_path $TARGET_FOLDER/tokenizer.model --seed $seed --max_seq_len 2048 --max_gen_len 2048 --count 0 | tee -a ${size}_startrek.txt
done
@cedrickchee
cedrickchee / nodejs-good-practices.md
Last active April 10, 2023 13:02
Node.js Good Practices and Mastering Node.js
View nodejs-good-practices.md

Node.js Good Practices and Mastering Node.js

Status: Draft

Although Node.js has many great frameworks, starter kits, and boilerplates, they were never meant to be production ready immediately. Improve your Node.js knowledge and skills. This post aims to bridge the gap.

@cedrickchee
cedrickchee / notion-ai-internals.md
Created January 5, 2023 05:54
Notion AI Internals
View notion-ai-internals.md

Notion AI Internals

(This is a fleeting notes. Writing is in progress. INCOMPLETE.)

notion-ai-writing-editing_20230105_132213

notion-ai-writing-editing_20230105_132328

I got access to the public alpha of Notion AI on Dec 20 2022.

@cedrickchee
cedrickchee / LLMs.md
Last active May 16, 2023 08:53 — forked from yoavg/LLMs.md
Fix typos and grammar of the original writing.
View LLMs.md

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of ChatGPT, maybe played with it a little, and was impressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labor costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We