Skip to content

Instantly share code, notes, and snippets.

View Nova-Rift's full-sized avatar

Nova-Rift

  • San Francisco
View GitHub Profile
@benob
benob / reshard.py
Created March 4, 2023 17:35
Script to decompose/recompose LLAMA LLM models with different number of shards.
# script to decompose/recompose llama model in different number of shards
# note that it loads the full model * 2 in cpu memory
import os
import json
import sys
import torch
import glob
if len(sys.argv) != 4: