bugyangs

## README_hfd.md

      
              2 files
            
          
              26 forks
            
          
              26 comments
            
          
              120 stars
            
          
                padeoe
                / README_hfd.md
            
            
              Last active
              April 26, 2024 00:42
            
              
                CLI-Tool for download Huggingface models and datasets with aria2/wget+git
              
          
    🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.
Features


⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.


## README-Template.md

      
              1 file
            
          
              4763 forks
            
          
              1700 comments
            
          
              13988 stars
            
          
                PurpleBooth
                / README-Template.md
            
            
              Last active
              April 26, 2024 17:22
            
              
                A template to make good README.md
              
          
    Project Title

One Paragraph of project description goes here
Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
Prerequisites