mtisz

import numpy as np
import torch
import jax
from tqdm import tqdm

from model import LanguageModelConfig, TransformerConfig, QuantizedWeight8bit as QW8Bit
from runners import InferenceRunner, ModelRunner, sample_from_model

CKPT_PATH = "./checkpoints"

mtisz

Fork of qlora: https://github.com/jondurbin/qlora

Make sure to change dataset format, and dataset path to your file, along with model/output paths.

If you want to modify the prompt format, edit this: https://github.com/jondurbin/qlora/blob/main/qlora.py#L433

Args used:

python qlora.py \