Skip to content

Instantly share code, notes, and snippets.

@epicfilemcnulty
epicfilemcnulty / 2safetensors.py
Created September 13, 2023 20:21
convert pytorch weights to safetensors
import json
import os
import sys
from collections import defaultdict
from tqdm import tqdm
import argparse
import torch
from safetensors.torch import load_file, save_file
@rain-1
rain-1 / llama-home.md
Last active June 19, 2024 03:05
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@yamaya
yamaya / xcode-clang-vers
Last active June 21, 2024 08:25
Xcode clang version record
# Xcode 4.3.3
Apple clang version 3.1 (tags/Apple/clang-318.0.61) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin11.4.0
Thread model: posix
# Xcode 4.3.2
Apple clang version 3.1 (tags/Apple/clang-318.0.58) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin11.4.0
Thread model: posix
@rosenfeld
rosenfeld / chef-gitorious-etc-solo.rb
Created February 28, 2011 12:29
Chef configuration files for Gitorious
file_cache_path "/root/chef-solo"
cookbook_path "/root/chef-solo/cookbooks"
json_attribs "/root/chef-solo/node.json"