fakerybakery

Tips for Axolotl Training

Adding ChatML Tokens

special_tokens:
  bos_token: "<s>"
  eos_token: "</s>"
  unk_token: "<unk>"
tokens:

fakerybakery

Get Flash Attention + Axolotl + Torch to work on CUDA 12.1

This fixed it for me

pip uninstall -y torch flash-attn
conda install pytorch==2.1.2 torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
pip install -U git+https://github.com/Dao-AILab/flash-attention

fakerybakery

How to convert Mixtral 8x22B to HF

You'll need:

• 600GB disk space (the model itself is ~280, then you'll convert it to HF which will double it)
• High bandwidth

First, download the Mixtral model using a torrent client:

magnet:?xt=urn:btih:9238b09245d0d8cd915be09927769d5f7584c1c9&dn=mixtral-8x22b&tr=udp%3A%2F%http://2Fopen.demonii.com%3A1337%2Fannounce&tr=http%3A%2F%http://2Ftracker.opentrackr.org%3A1337%2Fannounce

fakerybakery

<?php
# License:   MPL 2.0 (Mozilla Public License, version 2.0)
# Copyright: Copyright (c) 2023 mrfakename <mrfake.name>. All rights reserved, except as licensed under the Mozilla Public License, version 2.0
$ptext = "Progress: |                      |";
echo $ptext;
for ($percent = 0 ; $percent <= 100 ; $percent++) {
    echo "\033[" . strlen($ptext) . "D";
    $x = round(($percent / 10) * 2);
    echo "Progress: | " . str_repeat("•", $x) . str_repeat(" ", 20 - $x) . " |";
    usleep(50000);

fakerybakery

FILE_CONTENTS = 'File Contents'
REPO_TYPE = 'dataset'
REPO_ID = 'username/repo'
HF_TOKEN = 'hf_***'

import requests
import json
import base64

x = [

fakerybakery

MergeKit Example on Colab w/ Less Memory

Copyright (c) 2023. All rights reserved. Redistribution in any form is prohibited, however linking to this Gist is permitted.

fakerybakery

# License:      Unlicense, attribution optional but appreciated
# Copyright:    Copyright (c) 2023 mrfakename. All rights reserved. Distributed under the Unlicense license.
# Author:       mrfakename
# Date created: Dec 23, 2023
# Published on: GitHub Gist

import torch

def get_device():
	if torch.cuda.is_available():

fakerybakery

How to (really) uninstall MacPorts (macOS ONLY)

Ready to ditch MacPorts? Here's how to (really) do it:

If you follow the MacPorts Uninstallation Guide, which I really recommend you try first, you may encounter the following error:

zsh: no matches found: ...

fakerybakery

(Must be done on Desktop)

Find two coordinates on Google Maps.

Right click the first point, click the coordinates. Paste them into the code.

Right click on the second point, click the coordinates. Paste them into the code.

Right click the first point again, click Measure Distance. Then click the second point. Then run the code.

fakerybakery

Make a git repo with 10M commits

Just run sh run.sh

License: Please credit this if you publish a repo made with this