Skip to content

Instantly share code, notes, and snippets.

View pstaender's full-sized avatar
👾
extra terrestrial

Philipp Staender pstaender

👾
extra terrestrial
  • Cologne, Germany
View GitHub Profile
@rain-1
rain-1 / llama-home.md
Last active May 16, 2024 04:58
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@dhh
dhh / Gemfile
Created June 24, 2020 22:23
HEY's Gemfile
ruby '2.7.1'
gem 'rails', github: 'rails/rails'
gem 'tzinfo-data', '>= 1.2016.7' # Don't rely on OSX/Linux timezone data
# Action Text
gem 'actiontext', github: 'basecamp/actiontext', ref: 'okra'
gem 'okra', github: 'basecamp/okra'
# Drivers
  1. apt install bbswitch
  2. /etc/modprobe.d/bbswitch.conf
        options bbswitch load_state=0 unload_state=1
    To run bbswitch without bumblebeed on system startup, do not forget to add bbswitch to /etc/modules-load.d.
    nano /etc/modules-load.d
        bbswitch
I was drawn to programming, science, technology and science fiction
ever since I was a little kid. I can't say it's because I wanted to
make the world a better place. Not really. I was simply drawn to it
because I was drawn to it. Writing programs was fun. Figuring out how
nature works was fascinating. Science fiction felt like a grand
adventure.
Then I started a software company and poured every ounce of energy
into it. It failed. That hurt, but that part is ok. I made a lot of
mistakes and learned from them. This experience made me much, much
@lizthegrey
lizthegrey / attributes.rb
Last active February 24, 2024 14:11
Hardening SSH with 2fa
default['sshd']['sshd_config']['AuthenticationMethods'] = 'publickey,keyboard-interactive:pam'
default['sshd']['sshd_config']['ChallengeResponseAuthentication'] = 'yes'
default['sshd']['sshd_config']['PasswordAuthentication'] = 'no'
@mosquito
mosquito / README.md
Last active May 19, 2024 11:04
Add doker-compose as a systemd unit

Docker compose as a systemd unit

Create file /etc/systemd/system/docker-compose@.service. SystemD calling binaries using an absolute path. In my case is prefixed by /usr/local/bin, you should use paths specific for your environment.

[Unit]
Description=%i service with docker compose
PartOf=docker.service
After=docker.service
@nicksantamaria
nicksantamaria / fork-example.php
Created October 20, 2016 22:35
Example: Parallel processing in PHP using pcntl_fork()
<?php
/**
* @file
* Basic demonstration of how to do parallel threads in PHP.
*/
// This array of "tasks" could be anything. For demonstration purposes
// these are just strings, but they could be a callback, class or
// include file (hell, even code-as-a-string to pass to eval()).