Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active June 19, 2024 03:05
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@JaimeStill
JaimeStill / AppDbContext.cs
Last active July 9, 2024 04:38
ASP.NET Core Active Directory Identity Integration Middleware
// User entity class added for convenience. All other entities removed for clarity
namespace App.Data
{
public class AppDbContext : DbContext
{
public AppDbContext(DbContextOptions<AppDbContext> options) : base(options) { }
public DbSet<User> Users { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
@adamhaney
adamhaney / dag.py
Created June 14, 2017 18:10
DBT Airflow DAG with model/graph introspection
from datetime import datetime, timedelta
import networkx as nx
from airflow import DAG
from airflow.operators import BashOperator, SubDagOperator
start_date = datetime(year=2017, month=6, day=13, hour=19, minute=0)
schedule_interval = '0 * * * 1-5'
default_args = {
00001 PF.Magic's Petz 3 Player
00002 Headspace Beatnik Player Stub V1.0.0.1
00003 ZingWeb Plug-in for Netscape
00004 Windows Media Player Plug-in
00005 Parable ThingViewer 1.0
00006 Cosmo Player 2.1
00007 Adobe Acrobat
00008 Microsoft ActiveX Portability Container (NPWrap)
00009 VDOLive small plugin
00010 RealPlayer G2 LiveConnect (32-bit)
@edeustace
edeustace / sp.AutoGenerateAuditTableAndTrigger.sql
Created April 13, 2012 15:01
A stored procedure for sqlserver 2008 to generate history tables and triggers
/************************************************************************************************************
Inspired by: http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=84331
Created By: Bryan Massey
Created On: 3/11/2007
Comments: Stored proc performs the following actions:
1) Queries system tables to retrieve table schema for @TableName parameter
2) Creates a History table ( @TableName + '_History') to mimic the original table, plus include
additional history columns.
3) If @CreateTrigger = 'Y' then it creates an Update/Delete trigger on the @TableName table,
which is used to populate the History table.