Skip to content

Instantly share code, notes, and snippets.

@mmnga
mmnga / convert_shisa.py
Created December 10, 2023 12:11
Model: Conversion script in llama.cpp for gguf in augmxnt/shisa-7b-v1
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import concurrent.futures
import enum
import faulthandler
import functools
import itertools
import json
@mmnga
mmnga / convert-hf-to-gguf_watashiha_gpt.py
Created December 29, 2023 18:14
watashiha-gpt-6bのgguf変換コード byteのtokenが無いので使ってなさそうな絵文字の一部をbyteトークンに再割り当てして対応
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import contextlib
import json
import os
import re
import sys