d30a4d31b4f914757c1381166b9c683b

README.md

MergeKit Example on Colab w/ Less Memory

Copyright (c) 2023. All rights reserved. Redistribution in any form is prohibited, however linking to this Gist is permitted.

mergekit.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "private_outputs": true,
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/fakerybakery/d30a4d31b4f914757c1381166b9c683b\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Copyright (c) 2023 mrfakename. All rights reserved. Redistribution in any form is prohibited, however linking to [this Gist](https://gist.github.com/fakerybakery/d30a4d31b4f914757c1381166b9c683b) is permitted."
      ],
      "metadata": {
        "id": "TDh4WwdTZpx2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Download example configuration file and install MergeKit { display-mode: \"form\" }\n",
        "# @markdown This download an example configuration file. Make sure to edit it in the file browser.\n",
        "!wget https://gist.githubusercontent.com/fakerybakery/862e533af17e2cd1368ff592dd28669c/raw/config.yml\n",
        "!git clone https://github.com/cg123/mergekit\n",
        "!pip install -e mergekit"
      ],
      "metadata": {
        "id": "Y0peTQWtZeJz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Replace `config.yml` with your MergeKit config in the file browser or leave it as is for an example merge."
      ],
      "metadata": {
        "id": "TDh4WwdTZpx1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Note that setting the shard size to 1B will split the merged model into many chunks. However, this can be helpful for uploading large files. If you'd like to make the shard size even smaller, try 500M or 250M."
      ],
      "metadata": {
        "id": "g579L-HjZync"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "k9WdB4jCZaEV"
      },
      "outputs": [],
      "source": [
        "# @title Merge Model\n",
        "# @markdown This may take a while\n",
        "!mergekit-yaml \"config.yml\" mergedmodel --allow-crimes --out-shard-size 1B --lazy-unpickle"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now upload the model to Hugging Face. This is optional.\n",
        "\n",
        "1. Create a new model [here](https://huggingface.co/new)\n",
        "2. Run the code below to login. Make sure to login to HF in Colab first."
      ],
      "metadata": {
        "id": "Y1qapz79aAy8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pip install -U huggingface-hub\n"
      ],
      "metadata": {
        "id": "woB6ReoaaTE5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Upload to Hub (Make sure to create the model first) { display-mode: \"form\" }\n",
        "REPO_ID = 'username/model' # @param {type:\"string\"}\n",
        "from huggingface_hub import HfApi\n",
        "api = HfApi()\n",
        "api.upload_folder(\n",
        "    folder_path=\"mergedmodel\",\n",
        "    repo_id=REPO_ID,\n",
        "    repo_type=\"model\",\n",
        ")"
      ],
      "metadata": {
        "id": "0xS3yrZuaWzX"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}