Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active June 19, 2024 03:05
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@rain-1
rain-1 / LLM.md
Last active July 27, 2024 04:02
LLM Introduction: Learn Language Models

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

@ozbillwang
ozbillwang / Git_Behind_Proxy.md
Last active July 23, 2024 12:31
Configure Git to use a proxy (https or SSH+GIT)
@mamcx
mamcx / Lang.mkdown
Last active August 21, 2018 21:10
Lista de recursos sobre crear lenguajes de programacion
@CMCDragonkai
CMCDragonkai / memory_layout.md
Last active July 24, 2024 16:48
Linux: Understanding the Memory Layout of Linux Executables

Understanding the Memory Layout of Linux Executables

Required tools for playing around with memory:

  • hexdump
  • objdump
  • readelf
  • xxd
  • gcore
@staltz
staltz / introrx.md
Last active July 29, 2024 05:55
The introduction to Reactive Programming you've been missing
@jordanwade90
jordanwade90 / Slide
Created April 28, 2014 22:43
Use acme to switch between “slides” (actually just text files) in a directory. Blatantly stolen from Russ Cox (http://research.swtch.com/acme).
#!/usr/local/plan9/bin/rc
. /usr/local/plan9/lib/acme.rc
winname `{pwd}^/$1
winctl clean
winctl get
@wsargent
wsargent / docker_cheat.md
Last active June 29, 2024 19:32
Docker cheat sheet
@1wErt3r
1wErt3r / SMBDIS.ASM
Created November 9, 2012 22:27
A Comprehensive Super Mario Bros. Disassembly
;SMBDIS.ASM - A COMPREHENSIVE SUPER MARIO BROS. DISASSEMBLY
;by doppelganger (doppelheathen@gmail.com)
;This file is provided for your own use as-is. It will require the character rom data
;and an iNES file header to get it to work.
;There are so many people I have to thank for this, that taking all the credit for
;myself would be an unforgivable act of arrogance. Without their help this would
;probably not be possible. So I thank all the peeps in the nesdev scene whose insight into
;the 6502 and the NES helped me learn how it works (you guys know who you are, there's no