Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

View wrunk's full-sized avatar

Warren Runk wrunk

View GitHub Profile
@rain-1
rain-1 / llama-home.md
Last active February 22, 2024 05:52
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@raysan5
raysan5 / custom_game_engines_small_study.md
Last active April 23, 2024 13:41
A small state-of-the-art study on custom engines

CUSTOM GAME ENGINES: A Small Study

a_plague_tale

A couple of weeks ago I played (and finished) A Plague Tale, a game by Asobo Studio. I was really captivated by the game, not only by the beautiful graphics but also by the story and the locations in the game. I decided to investigate a bit about the game tech and I was surprised to see it was developed with a custom engine by a relatively small studio. I know there are some companies using custom engines but it's very difficult to find a detailed market study with that kind of information curated and updated. So this article.

Nowadays lots of companies choose engines like Unreal or Unity for their games (or that's what lot of people think) because d

@premek
premek / mv.sh
Last active March 5, 2024 17:43
Rename files in linux / bash using mv command without typing the full name two times
# Put this function to your .bashrc file.
# Usage: mv oldfilename
# If you call mv without the second parameter it will prompt you to edit the filename on command line.
# Original mv is called when it's called with more than one argument.
# It's useful when you want to change just a few letters in a long name.
#
# Also see:
# - imv from renameutils
# - Ctrl-W Ctrl-Y Ctrl-Y (cut last word, paste, paste)
@oxguy3
oxguy3 / RoughScroll.js
Last active September 27, 2023 15:02
Greasemonkey script for disabling smooth scrolling across the web (WIP! currently breaks a few websites, unfortunately)
// ==UserScript==
// @name RoughScroll
// @namespace http://tampermonkey.net/
// @version 0.1
// @description Disables smooth scrolling on ALL websites
// @author Hayden Schiff (oxguy3)
// @match *://*/*
// @grant none
// ==/UserScript==
/* jshint -W097 */
@vaskoz
vaskoz / programatic_tests.go
Created June 24, 2015 00:33
Run golang tests programatically
package main
import (
"flag"
"fmt"
"testing"
)
func Test1(t *testing.T) {
if 1+2 != 3 {
# Hello, and welcome to makefile basics.
#
# You will learn why `make` is so great, and why, despite its "weird" syntax,
# it is actually a highly expressive, efficient, and powerful way to build
# programs.
#
# Once you're done here, go to
# http://www.gnu.org/software/make/manual/make.html
# to learn SOOOO much more.
@chanks
chanks / gist:7585810
Last active February 29, 2024 03:50
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@ykessler
ykessler / timezones
Created August 14, 2012 14:50
JSON list of time zones (Based on Olson tz database)
[
{"group":"US (Common)",
"zones":[
{"value":"America/Puerto_Rico","name":"Puerto Rico (Atlantic)"},
{"value":"America/New_York","name":"New York (Eastern)"},
{"value":"America/Chicago","name":"Chicago (Central)"},
{"value":"America/Denver","name":"Denver (Mountain)"},
{"value":"America/Phoenix","name":"Phoenix (MST)"},
{"value":"America/Los_Angeles","name":"Los Angeles (Pacific)"},
{"value":"America/Anchorage","name":"Anchorage (Alaska)"},
@wrunk
wrunk / python_yaml_example.py
Created June 19, 2012 00:02
Quick python example of using yaml
'''
This is a quick reference example to get you started with yaml. Please review this,
but spend the time to fully read the wikipedia page on yaml:
http://en.wikipedia.org/wiki/Yaml
Note that the magic quote-less yaml parsing will still always strip values.
To avoid this you need to use quotes. Quotes can also force types to string type,
and are always valid (meaning omitting quotes is for convenience not necessity).
'''
@jboner
jboner / latency.txt
Last active April 22, 2024 15:20
Latency Numbers Every Programmer Should Know
Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns 3 us
Send 1K bytes over 1 Gbps network 10,000 ns 10 us
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD