Artefact2

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

jexp

Code along on motherduck.com

ATTACH 'md:_share/stackoverflow/6c318917-6888-425a-bea1-5860c29947e5'

• Blog Post Part 1: https://motherduck.com/blog/exploring-stackoverflow-with-duckdb-on-motherduck-1/
• Blog Post Part 2: https://motherduck.com/blog/exploring-stackoverflow-with-duckdb-on-motherduck-2/
• Free DDBIA Ebook: https://motherduck.com/duckdb-book

SELECT count(*)

markhng525

rain-1

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

• Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

ashvardanian

def read_matrix(filename: str, start_row: int = 0, count_rows: Optional[int] = None):
    """
    Read *.ibin, *.hbin, *.fbin, *.dbin files with matrixes.
    Args:
        :param filename (str): path to the matrix file
        :param start_row (int): start reading vectors from this index
        :param count_rows (int): number of vectors to read. If None, read all vectors
    Returns:
        Parsed matrix (numpy.ndarray)
    """

svpino

import re
import asyncio
from playwright.async_api import async_playwright


USERNAME = "TYPE YOUR USERNAME HERE"
PASSWORD = "TYPE YOUR PASSWORD HERE"
HOST = "zproxy.lum-superproxy.io:9222"

URL = "https://www.svpino.com/" # USE YOUR URL HERE

mfischr

#!/usr/bin/env bash
set -e

# pj-append.bash is a timestamped log file for you, a human.  Set up a cron job to launch it every
# hour to note what you were working on, or append lines from the terminal whenever you're chewing
# on a hard problem.
#
# Use the data to build a picture of what you worked on during the last week, or grep
# last quarter's log to find out why you decided to use library A instead of library B.
#

Narsil

import mmap
import torch
import json
import os
from huggingface_hub import hf_hub_download


def load_file(filename, device):
    with open(filename, mode="r", encoding="utf8") as file_obj:
        with mmap.mmap(file_obj.fileno(), length=0, access=mmap.ACCESS_READ) as m:

trygvebw

import torch
import numpy as np
import k_diffusion as K

from PIL import Image
from torch import autocast
from einops import rearrange, repeat

def pil_img_to_torch(pil_img, half=False):
    image = np.array(pil_img).astype(np.float32) / 255.0

karpathy

"""
stable diffusion dreaming
creates hypnotic moving videos by smoothly walking randomly through the sample space

example way to run this script:

$ python stablediffusionwalk.py --prompt "blueberry spaghetti" --name blueberry

to stitch together the images, e.g.:
$ ffmpeg -r 10 -f image2 -s 512x512 -i blueberry/frame%06d.jpg -vcodec libx264 -crf 10 -pix_fmt yuv420p blueberry.mp4