padeoe

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

• ⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
• 🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
• 🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
• 🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
• 🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.

padeoe

DDNS for Domain on Name.com

NAME.COM provide the API to create/update DNS record, we can implement DDNS without depending on any third party DDNS service.

• Python 3.8+ (because I used the latest := operator in 3.8, if you use 3.6 or others, you can modify the codes, very easy)
• v4 REST API of name.com
• Use ipinfo.io to get public IP

Step 1: Prepare a Proxy (Static IP)

Since we would like to setup DDNS, which means our server doesn't have a fixed IP, but the API of name.com requires the caller's IP whitelisted in advance, which means the caller's IP need to be fixed or limited in the number. However, the dynamic from ISP is not stable. So, a proxy server is required to solve the problem.

padeoe

功能特性

每隔 5 分钟执行一次网络系统登录检测，检查当前是否在线，如果不在线则登录上线。

• 定时检测
• 保存登录日志
• 用户名密码参数化

依赖 Python 3.

1. 创建登陆脚本

padeoe

{
    "domain": [
        "domain:nju.edu.cn",
        "domain:njuftp.org",
        "domain:ingentaconnect.com",
        "domain:nutrition.org",
        "domain:geoscienceworld.org",
        "domain:westlaw.com",
        "domain:agu.org",
        "domain:emeraldgrouppublishing.com",

padeoe

This script enhances the functionality of nvidia-smi and provides the following information:

• Username
• full process Command
• GPU ID
• PID

This is useful on multi-user servers and can be used to quickly identify which user is using the GPU and running what kind of program.

padeoe

FROM pytorch/pytorch:1.1.0-cuda10.0-cudnn7.5-devel

# use local apt mirror
RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g;s/security.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list
RUN apt-get update

# use local pip mirror
RUN mkdir ~/.pip \
    && printf '%s\n%s\n%s\n' '[global]' 'trusted-host = mirrors.aliyun.com' \
    'index-url = https://mirrors.aliyun.com/pypi/simple'>> ~/.pip/pip.conf

padeoe

[Script] Auto-Renew Wildcard Letsencrypt Certs for name.com

This tutorial can Auto-Renew wildcard letsencrypt certs for domain on name.com.

• wildcard domain, e.g., *.example.com
• domain hosted by https://www.name.com, we use the API of name.com to add DNS TXT records with a shell script.

Wildcard certificates issued by letsencrypt.org need DNS TXT record to challenge, we can add TXT record manually when you apply the cert. If we want to automate it, we need to write a script that use the API of DNS provider to add TXT record. certbot has provided command argument --manual-auth-hook to pass the script.

Step 1: Get a API token

Get your own api token provided by name.com: https://www.name.com/account/settings/api.

padeoe

# pull the latest image
check_update=`docker pull gitlab/gitlab-ce:latest`

# Check if newer image pulled
if [[ $check_update == *"Image is up to date for gitlab/gitlab-ce:latest"* ]]
then
    echo 'Gitlab is already up to date!'
else
    echo 'Start to update Gitlab...'

padeoe

FROM tensorflow/tensorflow:latest-gpu-py3

# 修改apt源
RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g;s/security.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list
RUN apt-get update

# 修改pip源
RUN mkdir ~/.pip \
    && printf '%s\n%s\n%s\n' '[global]' 'trusted-host = mirrors.aliyun.com' \
    'index-url = https://mirrors.aliyun.com/pypi/simple'>> ~/.pip/pip.conf

padeoe

//含"无用比较"的插入排序
void insertion_sort(int a[], int n) {
	for (int i = 0; i < n; i++) {
		for (int j = 0; j < i; j++) {
			if (a[i] < a[j]) {
				int tmp = a[j];
				a[j] = a[i];
				a[i] = tmp;
			}
		}