| { | |
| "schema_version": "v1", | |
| "name_for_model": "twilio", | |
| "name_for_human": "Twilio Plugin", | |
| "description_for_model": "Plugin for integrating the Twilio API to send SMS messages and make phone calls. Use it whenever a user wants to send a text message or make a call using their Twilio account.", | |
| "description_for_human": "Send text messages and make phone calls with Twilio.", | |
| "auth": { | |
| "type": "user_http", | |
| "authorization_type": "basic" | |
| }, |
Let's talk to an Alpaca-7B model using LangChain with a conversational chain and a memory window.
Install python packages using pip. Note that you need to install HuggingFace Transformers from source (GitHub) currently.
$ pip install git+https://github.com/huggingface/transformers
| # Based on: Original Alpaca Model/Dataset/Inference Code by Tatsu-lab | |
| import time, torch | |
| from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig | |
| tokenizer = LlamaTokenizer.from_pretrained("./checkpoint-1200/") | |
| def generate_prompt(instruction, input=None): | |
| if input: | |
| return f"""The following is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. |
The problem with large language models is that you can’t run these locally on your laptop. Thanks to Georgi Gerganov and his llama.cpp project, it is now possible to run Meta’s LLaMA on a single computer without a dedicated GPU.
There are multiple steps involved in running LLaMA locally on a M1 Mac after downloading the model weights.
Follow the instructions here under "Installation option 1: conda"
Download the desired Hugging Face converted model for LLaMA here
Copy the entire model folder, for example llama-13b-hf, into text-generation-webuimodels
info 9-3-23 Added 4bit LLaMA install instructions for cards as small as 6GB VRAM! (See "BONUS 4" at the bottom of the guide)
warning 9-3-23 Added Torrent for HFv2 Model Weights, required for ooga's webUI, Kobold, Tavern and 4bit (+4bit model)! Update ASAP!
danger 11-3-23 There's a new torrent version of the 4bit weights called "LLaMA-HFv2-4bit". The old "LLaMA-4bit" torrent may be fine. But if you have any issues with it, it's recommended to update to the new 4bit torrent or use the decapoda-research versions off of HuggingFace or produce your own 4bit weights. Newer Torrent Link or [Newer Magnet Link](magnet:?xt=urn:btih:36945b5958b907b3ab69e963ba0de1abdf48c16c&dn=LLaMA-HFv2-4bit&tr=http%3a%2f%2fbt1.archive.org%3a6969%2fannounce&tr=http%3a%2f%2fbt2.archive.org%3a696
| mp=1; size=7B; # to run 7B | |
| mp=8; size=65B; # to run 65B | |
| for seed in $(randint 1000000) | |
| do | |
| export TARGET_FOLDER=~/ml/data/llama/LLaMA | |
| time python3 -m torch.distributed.run --nproc_per_node $mp example.py --ckpt_dir $TARGET_FOLDER/$size --tokenizer_path $TARGET_FOLDER/tokenizer.model --seed $seed --max_seq_len 2048 --max_gen_len 2048 --count 0 | tee -a ${size}_startrek.txt | |
| done |
Status: Draft
Although Node.js has many great frameworks, starter kits, and boilerplates, they were never meant to be production ready immediately. Improve your Node.js knowledge and skills. This post aims to bridge the gap.
(This is a fleeting notes. Writing is in progress. INCOMPLETE.)
I got access to the public alpha of Notion AI on Dec 20 2022.
Audience: I assume you heard of ChatGPT, maybe played with it a little, and was impressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.
Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labor costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We