Skip to content

Instantly share code, notes, and snippets.

@lexaone
lexaone / llama-home.md
Created May 14, 2023 20:20 — forked from rain-1/llama-home.md
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@lexaone
lexaone / Synology-Diskstation-Git.md
Created October 9, 2022 09:11 — forked from walkerjeffd/Synology-Diskstation-Git.md
Instructions for setting up git server on Synology Diskstation

Configure Synology NAS as Git Server

Instructions for setting up a git server on a Synology NAS with Diskstation. Specifically, I am using a DS414 with DSM 5.0.

Set Up User and Folder

  • Create user gituser via Diskstation interface (with File Station and WebDAV privilages)
  • Add new shared folder called git (located at /volume1/git) with read/write access for gituser and admin. This folder will hold all the repos.
  • Install Git Server package via Diskstation
@lexaone
lexaone / key_light
Created September 4, 2022 22:38
This script change cyclically ite8291 keyboard brightness using ite8291r3-ctl utility
#!/usr/bin/env bash
set -o nounset
set -o errexit
set -o errtrace
set -o pipefail
BRIGHTNESS_STEP=10 #variable
BRIGHTNESS_MAX=50 #constant
ITE_UTIL='/usr/bin/ite8291r3-ctl'
#!/bin/sh
case $1/$2 in
pre/*)
echo "Going to $2..."
# Place your pre suspend commands here, or `exit 0` if no pre suspend action required
;;
post/*)
echo "Waking up from $2..."
#ite8291r3-ctl effect ripple -s 1 -b 50 -r --save
/usr/bin/ite8291r3-ctl anim --f /home/lexa/src/ite8291r3-ctl/assets/animations/manjaro_lexa
/*
Coloring the Fn keys for keyboard brightness, volume, display brightness and the i3 mod keys.
F-Keys and Fn-key white.
All other keys in manajaro blue-green.
Created for:
- XMG Fusion 15 (Intel Whitebook QC71)
- QWERTZ keyboard layout
- 6 lines, 16 columns of keys
set nocompatible " disable compatibility to old-time vi
set showmatch " show matching brackets.
"set ignorecase " case insensitive matching
set mouse=v " middle-click paste with mouse
set hlsearch " highlight search results
set tabstop=4 " number of columns occupied by a tab character
set softtabstop=4 " see multiple spaces as tabstops so <BS> does the right thing
set expandtab " converts tabs to white space
set shiftwidth=4 " width for autoindents
set autoindent " indent a new line the same amount as the line just typed
#!/bin/sh
# load dependency modules
###!!!this is patched version of script, we use only half of CPU cores (4 instead of 8) and quarter of memory (instead of half) 8 of 32
CPU_CORES=$(grep -c ^processor /proc/cpuinfo | sed 's/^0$/1/')
NRDEVICES=$((CPU_CORES / 2))
if modinfo zram | grep -q ' zram_num_devices:' 2>/dev/null; then
MODPROBE_ARGS="zram_num_devices=${NRDEVICES}"
const std = @import("std");
// my personal preferences (uncomment it if need)
//const process = std.process;
//const ProgErr = error{
// ArgsErr,
//};
pub fn main() anyerror!void {
//var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
//defer arena.deinit();
@lexaone
lexaone / zig shell trick
Last active September 9, 2021 17:47
zig shell trick
//usr/bin/env zig run $0 $@ ; exit
const std = @import("std");
pub fn main() !void {
std.log.info("Awesome, it works\n", .{});
}
######################################################################
# thinkfan 0.7 example config file
# ================================
#
# ATTENTION: There is only very basic sanity checking on the configuration.
# That means you can set your temperature limits as insane as you like. You
# can do anything stupid, e.g. turn off your fan when your CPU reaches 70°C.
#
# That's why this program is called THINKfan: You gotta think for yourself.
#