Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active June 24, 2025 11:12
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@avillp
avillp / Unportify-v1.4.3.js
Last active November 12, 2023 15:22
Unportify helps you export your Google Play Music playlists.
/*
Unportify is a script that exports your Google Play music to text.
Copyright (C) 2016 Arnau Villoslada
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
@hereswhatidid
hereswhatidid / Media.php
Last active December 11, 2019 08:50
PrestaShop Media class override to allow for forcing some inline JavaScripts to remain inline.
<?php
Class Media extends MediaCore
{
public static function deferScript($matches)
{
if (!is_array($matches))
return false;
$inline = '';
if (isset($matches[0]))
@phirefly
phirefly / gist:7120713
Created October 23, 2013 15:19
Basic bootstrap 3 nav in haml
%nav.navbar.navbar-default{role: "navigation"}
/ Brand and toggle get grouped for better mobile display
.navbar-header
%button.navbar-toggle{"data-target" => ".navbar-ex1-collapse", "data-toggle" => "collapse", type: "button"}
%span.sr-only Toggle navigation
%span.icon-bar
%span.icon-bar
%span.icon-bar
%a.navbar-brand{href: "#"} Brand
/ Collect the nav links, forms, and other content for toggling