Skip to content

Instantly share code, notes, and snippets.

View leavyli's full-sized avatar
💭
I may be slow to respond.

leavyli

💭
I may be slow to respond.
View GitHub Profile
@leavyli
leavyli / longest_chinese_tokens_gpt4o.py
Created May 15, 2024 15:00 — forked from ctlllll/longest_chinese_tokens_gpt4o.py
Longest Chinese tokens in gpt4o
import tiktoken
import langdetect
T = tiktoken.get_encoding("o200k_base")
length_dict = {}
for i in range(T.n_vocab):
try:
length_dict[i] = len(T.decode([i]))
except:
@leavyli
leavyli / README_hfd.md
Created December 29, 2023 01:07 — forked from padeoe/README_hfd.md
CLI Tool for Downloading Huggingface Models and Datasets

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

  • ⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
  • 🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
  • 🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., .bin and .safetensors).
  • 🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
  • 🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.
// ==UserScript==
// @name aicompasspro的自动循环发音
// @namespace ...
// @version 0.1.0
// @description aicompasspro的自动循环发音
// @author You
// @match https://m.openlanguage.com/*
// @match https://www.aicompasspro.com/*
// @grant none
// @run-at document-idle
@leavyli
leavyli / php
Last active March 2, 2018 15:24
替换手机星号
<?php
//输出结果string(115) "mobile: 13929****47,13929****47,13929****47,13929****47,1****3,13929****47,13929****47, name:liwei, moblie: 1****6"
$s = "mobile: 13929571847,13929527847,13929517847,13929575847,123,13929575847,13929575847, name:liwei, moblie: 12636";
$p =[
"/(.*?)(\d)(\d{1,4})(\d{1,2},)/",
"/(.*?)(\d)(\d{1,4})(\d{1,2}$)/",
];
$r = [
'\1\2****\4',
'\1\2****\4',
@leavyli
leavyli / run_test.bat
Created January 6, 2018 11:58
Yii2:unit:test:command:run:yii2测试的命令例子
#### 执行单个类的测试
.\vendor\bin\codecept run unit models\UserTest
#### 执行类的某个方法的测试
.\vendor\bin\codecept run unit models\UserTest:testFindUserById
@leavyli
leavyli / mklink.bat
Created January 6, 2018 11:53
windows:cmd:mklink:windows下创建软硬链接的命令
@leavyli
leavyli / selenium_test
Created January 6, 2018 11:49
python:selenium:verity_by_slide:(一个简单的通过selenium控制滑动验证的例子,网站可能未必能用)
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.common.exceptions import NoSuchElementException
import time
import os
import base64
import re
import uuid
# 初始化chrome的dirver路径
@leavyli
leavyli / read_headers_form_file.py
Last active January 6, 2018 11:40
python:web:headers:read from file:从文件读取请求头,多用在调试爬虫的时候,浏览器的请求头直接复制到文件读出来。
headers = {}
with open(r"e:/test/header") as f:
s = f.read()
for i in s.split("\n"):
k = i.split(":")[0]
v = i.split(":")[1]
headers[k.strip()] = v.strip()
@leavyli
leavyli / _bootstrap.php
Last active January 6, 2018 11:55
Yii2:unit:_bootstrap.php:用于单元测试的时候配置命名空间,从而可以加载对应的类
<?php
// add unit testing specific bootstrap code here
$classLoader = new \Composer\Autoload\ClassLoader();
$classLoader->addPsr4("tests\\components\\", __DIR__ . "/components", false);
$classLoader->addPsr4("tests\\models\\", __DIR__ . "/models", false);
$classLoader->register();