Skip to content

Instantly share code, notes, and snippets.

View 0x0177b11f's full-sized avatar

tiansheng li 0x0177b11f

25 followers · 17 following
View GitHub Profile
@0x0177b11f
0x0177b11f / chunking-regex.ts
Created August 15, 2024 03:26
Use regex to do chunking by using all semantic cues
// Used in https://jina.ai/tokenizer (Aug. 14th version)
// Define variables for magic numbers
const MAX_HEADING_LENGTH = 6;
const MAX_HEADING_CONTENT_LENGTH = 200;
const MAX_HEADING_UNDERLINE_LENGTH = 200;
const MAX_HTML_HEADING_ATTRIBUTES_LENGTH = 100;
const MAX_LIST_ITEM_LENGTH = 200;
const MAX_NESTED_LIST_ITEMS = 5;
const MAX_LIST_INDENT_SPACES = 7;
const MAX_BLOCKQUOTE_LINE_LENGTH = 200;
@0x0177b11f
0x0177b11f / Debian.Dockerfile
Last active November 7, 2020 01:37
debian preconfig dockerfile
FROM debian:buster
ENV DEBIAN_FRONTEND=noninteractive
RUN set -e -x; \
apt-get update; \
apt-get install -y locales locales-all sudo apt-utils apt-transport-https ca-certificates curl; \
useradd -m -s /bin/bash builder; \
usermod -a -G sudo builder; \
echo 'builder ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/builder; \
@0x0177b11f
0x0177b11f / testReg_test.go
Created June 19, 2020 11:41
golang get json field Benchmark (regexp vs json library)
package main
import (
"encoding/json"
"time"
"math/rand"
"regexp"
"strconv"
@0x0177b11f
0x0177b11f / rm_git_tmp_pack.sh
Created March 25, 2020 00:32
delete tmp_pack files
# no confirmation
find -name "tmp_pack*" -type f -exec rm {} \;
# with confirmation from user
find -name "tmp_pack*" -type f -exec rm -i {} \;
@0x0177b11f
0x0177b11f / vps-startup.sh
Last active February 22, 2025 07:48
vps startup script
#!/bin/sh
cat > /etc/modules-load.d/netfilter.conf <<EOF
br_netfilter
EOF
cat > /etc/sysctl.d/00-panic.conf <<EOF
kernel.panic=10
EOF
@0x0177b11f
0x0177b11f / nofile.conf
Created October 6, 2019 13:59
limits.d
* soft nofile 655350
* hard nofile 1310700
@0x0177b11f
0x0177b11f / 60-scheduler.rules
Created August 21, 2019 00:14
60-scheduler.rules
ACTION=="add|change", KERNEL=="sd[a-z]|mmcblk[0-9]*", ATTR{queue/scheduler}="bfq"
ACTION=="add|change", KERNEL=="nvme[0-9]n[0-9]", ATTR{queue/scheduler}="bfq"
@0x0177b11f
0x0177b11f / zram-swap@.service
Last active May 19, 2020 03:40
zram swap generator
[Unit]
Description=zram swap generator on %I
After=multi-user.target
[Service]
Type=oneshot
RemainAfterExit=true
TimeoutSec=60
Environment=ALGO=zstd
@0x0177b11f
0x0177b11f / grub.config
Created May 15, 2019 01:27
amd_iommu_kernel_args
GRUB_CMDLINE_LINUX="amd_iommu=on iommu=pt rhgb quiet"
@0x0177b11f
0x0177b11f / bridged-forward.sh
Created May 1, 2019 12:00
firewalld forward bridged
sudo firewall-cmd --permanent --direct --passthrough ipv4 -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
sudo firewall-cmd --permanent --direct --passthrough ipv6 -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT