Skip to content

Instantly share code, notes, and snippets.

View strong-code's full-sized avatar

Colin strong-code

  • California
View GitHub Profile
@rain-1
rain-1 / llama-home.md
Last active June 24, 2025 11:12
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
#
# A sorting algorithm that leverages the "green" threads of Ruby's Thread
# class. It uses a timer to count down from all positive integer elements
# of the supplied array, and passes them to a sorted array once the timer
# has expired. Since in theory integers of the same value can have their
# timers finish at the same time or at arbitrarily different times (in
# terms of which finishes first), this sorting is not considered stable.
#
# In theory, this algorithm runs in O(n*t) time where n is the number of
# elements in the supplied array and t is the unit of time used for the
@strong-code
strong-code / String Formatting
Created August 22, 2013 04:56
JS String Formatting function
/*C# style string formatting function
var greet = String.format("Hello, {0}", name);
*/
String.format = function() {
var s = arguments[0];
for (var i = 0; i < arguments.length - 1; i++) {
var reg = new RegExp("\\{" + i + "\\}", "gm");
@piscisaureus
piscisaureus / pr.md
Created August 13, 2012 16:12
Checkout github pull requests locally

Locate the section for your github remote in the .git/config file. It looks like this:

[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = git@github.com:joyent/node.git

Now add the line fetch = +refs/pull/*/head:refs/remotes/origin/pr/* to this section. Obviously, change the github url to match your project's URL. It ends up looking like this: