Skip to content

Instantly share code, notes, and snippets.

@bjj
bjj / README_hfd.md
Last active February 15, 2025 17:55 — forked from SinclairCoder/README_hfd.md
Command-line Tool for Easy Downloading of Huggingface Models

🤗Huggingface Model Downloader

FORK: Supports --tag for downloading tags other than main

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

  • ⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
  • 🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
  • 🚫 File Exclusion: Use --exclude to skip specific files, save time for models with duplicate formats (e.g., .bin and .safetensors).
  • 🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
的 0.0588839 0.0588839
是 0.0205837 0.0794676
这个 0.0184868 0.0979545
你 0.017203 0.115157
它 0.0162188 0.131376
了 0.0156197 0.146996
一 0.0142931 0.161289
个 0.0127097 0.173999
把 0.0125813 0.18658
可以 0.0122818 0.198862
的 0.046723 0.046723
我 0.0409963 0.0877193
了 0.0315426 0.119262
你 0.0309972 0.150259
是 0.0186347 0.168894
小白 0.0144532 0.183347
啊 0.0136351 0.196982
就 0.00845378 0.205436
好 0.00809017 0.213526
雨 0.00781747 0.221344