Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active May 16, 2024 04:58
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@kconner
kconner / macOS Internals.md
Last active May 17, 2024 06:10
macOS Internals

macOS Internals

Understand your Mac and iPhone more deeply by tracing the evolution of Mac OS X from prelease to Swift. John Siracusa delivers the details.

Starting Points

How to use this gist

You've got two main options:

@ddanier
ddanier / fastapi_globals.py
Last active May 15, 2024 19:53
flask.g for FastAPI.
"""
This allows to use global variables inside the FastAPI application using async mode.
# Usage
Just import `g` and then access (set/get) attributes of it:
```python
from your_project.globals import g
@JoeyBurzynski
JoeyBurzynski / 55-bytes-of-css.md
Last active May 15, 2024 20:11
58 bytes of css to look great nearly everywhere

58 bytes of CSS to look great nearly everywhere

When making this website, i wanted a simple, reasonable way to make it look good on most displays. Not counting any minimization techniques, the following 58 bytes worked well for me:

main {
  max-width: 38rem;
  padding: 2rem;
  margin: auto;
}
@mhoyer
mhoyer / single-user-msys-sshd.md
Last active March 11, 2024 09:22
Install single-user SSH daemon as Windows service with pure msys tools

Install single-user SSH daemon as Windows service with pure msys tools

WARNING This approach does not enable impersonation. Thus, only a single user account is able to connect through ssh - the same user we use for running the Windows service. At time of writing it seem to only be possible with cygwin or msys2 to achieve multi-user support for sshd under Windows. E.g. https://ghc.haskell.org/trac/ghc/wiki/Building/Windows/SSHD

Requirements

  • Latest Git for Windows. I tried it with v2.6.0. (Note: with v2.5.3 sshd tried to load authorized keys from /c/.ssh instead of ~/.ssh).

Create unprivileged sshd user

@ragingwind
ragingwind / Backend Architectures Keywords and References.md
Last active April 17, 2024 10:51
Backend Architectures Keywords and References
@mrchief
mrchief / LICENSE.md
Last active March 23, 2024 12:28
Add "Open with Sublime Text 2" to Windows Explorer Context Menu (including folders)

MIT License

Copyright (c) [year] [fullname]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

@Deco
Deco / deepcopy.lua
Created October 31, 2012 05:38
Lua Non-recursive Deep-copy
--[[ deepcopy.lua
Deep-copy function for Lua - v0.2
==============================
- Does not overflow the stack.
- Maintains cyclic-references
- Copies metatables
- Maintains common upvalues between copied functions (for Lua 5.2 only)
TODO
@spion
spion / a-warning.md
Last active March 25, 2024 03:01
C++ versus V8 versus luajit versus C benchmark - (hash) tables

Warning

This benchmark has been misleading for a while. It was originally made to demonstrate how JIT compilers can do all sorts of crazy stuff to your code - especially LuaJIT - and was meant to be a starting point of discussion about what exactly LuaJIT does and how.

As a result, its not indicative of what its performance may be on more realistic data. Differences can be expected because

  1. the text will not consist of hard-coded constants
@emk
emk / apache-logs-hive.sql
Created January 3, 2012 18:01
Apache log analysis with Hadoop, Hive and HBase
-- This is a Hive program. Hive is an SQL-like language that compiles
-- into Hadoop Map/Reduce jobs. It's very popular among analysts at
-- Facebook, because it allows them to query enormous Hadoop data
-- stores using a language much like SQL.
-- Our logs are stored on the Hadoop Distributed File System, in the
-- directory /logs/randomhacks.net/access. They're ordinary Apache
-- logs in *.gz format.
--
-- We want to pretend that these gzipped log files are a database table,