padeoe

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

• ⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
• 🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
• 🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
• 🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
• 🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.

taskylizard

Some stats: - Total number of links: 23983

---

---

◄◄ Back to Wiki Index

---

---

► System Tools

veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

kalomaze

Kalomaze's Local LLM Glossary

Not super comprehensive (yet), but I think having up to date documentation like this should be quite helpful for those out of the loop. Things change all the time in local AI circles, and it can be dizzying to catch up from an outsider's perspective, especially if you are new to the more technical aspects of language models in general (and not just locally hosted LLMs).

Available Models

Llama

• A language model series created by Meta. Llama 1 was originally leaked in February 2023; Llama 2 then officially released later that year with openly available model weights & a permissive license. Kicked off the initial wave of open source developments that have been made when it comes to open source language modeling. The Llama series comes in four distinct sizes: 7b, 13b, 34b (only Code Llama was released for Llama 2 34b), and 70b. As of writing, the hotly anticipated Llama 3 has yet to arrive.

Mistral

• Mistral AI is a French company that also distributes open weight

DenverCoder1

Script for converting watched movies and Wishlist CSVs from MovieLens to Letterboxd format.

Steps:

1. Download movielens-ratings.csv and movielens-logs.csv from https://movielens.org/profile/settings/import-export by clicking "export ratings" and "export activity logs". For watchlist import, also download movielens-wishlist.csv by clicking "export wishlist".
2. Change the RATINGS_CSV, LOGS_CSV, and WISHLIST_CSV to the paths of the movielens-ratings.csv, movielens-logs.csv, and movielens-wishlist.csv files respectively.

subfuzion

There are certain files created by particular editors, IDEs, operating systems, etc., that do not belong in a repository. But adding system-specific files to the repo's .gitignore is considered a poor practice. This file should only exclude files and directories that are a part of the package that should not be versioned (such as the node_modules directory) as well as files that are generated (and regenerated) as artifacts of a build process.

All other files should be in your own global gitignore file:

• Create a file called .gitignore in your home directory and add any filepath patterns you want to ignore.
• Tell git where your global gitignore file is.

Note: The specific name and path you choose aren't important as long as you configure git to find it, as shown below. You could substitute .config/git/ignore for .gitignore in your home directory, if you prefer.

mathix420

// ==UserScript==
// @name        Medium Paywall Bypass
// @namespace   Violentmonkey Scripts
// @run-at      document-start
// @match       *://*.medium.com/*
// @match       *://medium.com/*
// @match       *://*/*
// @grant       none
// @version     2.3
// @inject-into content

imba-tjd

云

IaaS指提供系统（可以自己选）或者储存空间之类的硬件，软件要自己手动装；PaaS提供语言环境和框架（可以自己选）；SaaS只能使用开发好的软件（卖软件本身）；BaaS一般类似于非关系数据库，但各家不通用，有时还有一些其它东西。

其他人的集合

• https://education.github.com/pack GitHub学生包，需用教育邮箱验证。各种福利，可从DigitalOcean上手
• https://github.com/ripienaar/free-for-dev 本文尽量不与此项目重复
• https://free.zhelper.net/
• https://github.com/AchoArnold/discount-for-student-dev

mingfeima

(Internal Tranining Material)

Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:

• How to find the bottleneck operator?
• How to trace source file of a particular operator?
• How do I indentify threading issues? (oversubscription)
• How do I tell a specific operator is running efficiently or not?

This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.

tvytlx

// 便签元素在 frame 里，不能直接用 artoo 处理，得先得到内部的 dom 元素，然后传给 artoo。
const iframe = 'iframe#js_note_mod_ctn.js_sandbox.business-mod-ctn.note-mod-ctn';
let get_container = (str)=>{return $(iframe)[0].contentDocument.body.querySelector(str)};
let notes_container = get_container('.home-bd .briefs-ctn.js_home_briefs_ctn');

// 开始抓取
let is_in_box = (this_)=>{
    return (this_.attr('class')=='js_folder_brief folder-brief js_normal_folder js_lock') ||
    (this_.attr('class')=='js_folder_brief folder-brief js_normal_folder')
};