Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active February 22, 2024 05:52
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@kconner
kconner / macOS Internals.md
Last active April 22, 2024 21:28
macOS Internals

macOS Internals

Understand your Mac and iPhone more deeply by tracing the evolution of Mac OS X from prelease to Swift. John Siracusa delivers the details.

Starting Points

How to use this gist

You've got two main options:

@nanxstats
nanxstats / gsDesign-gource.sh
Created October 28, 2022 02:13
Shell commands to generate version control visualization video for gsDesign using gource
# Clone repo
git clone https://github.com/keaven/gsDesign.git
cd gsDesign
# Run gource - this will generate a 411GB ppm file
gource -3840x2160 --seconds-per-day 0.1 --auto-skip-seconds 0.01 --file-idle-time 0 --font-size 34 --key --logo man/figures/logo.png -o gsDesign.ppm
# Convert ppm to mp4
ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i gsDesign.ppm -vcodec libx264 -preset medium -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 gsDesign.mp4
# Merge audio to video
ffmpeg -i gsDesign.mp4 -i music.mp3 -c:v copy -c:a aac output.mp4
# Recommended by YouTube
@slimsag
slimsag / ramblings.md
Last active December 13, 2023 08:02
Because cross-compiling binaries for Windows is easier than building natively

Because cross-compiling binaries for Windows is easier than building natively

I want Microsoft to do better, want Windows to be a decent development platform-and yet, I constantly see Microsoft playing the open source game: advertising how open-source and developer friendly they are - only to crush developers under the heel of the corporate behemoth's boot.

The people who work at Microsoft are amazing, kind, talented individuals. This is aimed at the company's leadership, who I feel has on many occassions crushed myself and other developers under. It's a plea for help.

The source of truth for the 'open source' C#, C++, Rust, and other Windows SDKs is proprietary

You probably haven't heard of it before, but if you've ever used win32 API bindings in C#, C++, Rust, or other languages, odds are they were generated from a repository called microsoft/win32metadata.

@Widdershin
Widdershin / ssr.md
Last active March 8, 2024 11:21
The absurd complexity of server-side rendering

In the olden days, HTML was prepared by the server, and JavaScript was little more than a garnish, considered by some to have a soapy taste.

After a fashion, it was decided that sometimes our HTML is best rendered by JavaScript, running in a user's browser. While some would decry this new-found intimacy, the age of interactivity had begun.

But all was not right in the world. Somewhere along the way, we had slipped. Our pages went uncrawled by Bing, time to first meaningful paint grew faster than npm, and it became clear: something must be done.

And so it was decided that the applications first forged for the browser would also run on the server. We would render our HTML using the same logic on the server and the browser, and reap the advantages of both worlds. In a confusing series of events a name for this approach was agreed upon: Server-side rendering. What could go wrong?

In dark rooms, in hushed tones, we speak of colours.

@huytd
huytd / wordle.md
Last active March 17, 2024 20:51
Wordle in less than 50 lines of Bash

image

How to use:

./wordle.sh

Or try the unlimit mode:

/* global __DEV__ */
import React, { useState, useEffect, useRef } from "react"
import {
facebookAppId,
facebookDisplayName,
iosOneSignalAppId,
androidOneSignalAppId,
sentryDsn,
} from "./app.json"
import { version } from "./package.json"