phiresky

/node_modules
*.log

phiresky

You can scale a SQLite database to multiple GByte in size and many concurrent readers by applying the below optimizations.

Run these every time you connect to the db

(some are applied permanently, but others are reset on new connection)

pragma journal_mode = WAL;

Instead of writing directly to the db file, write to a write-ahead-log instead and regularily commit the changes. Allows multiple concurrent readers, and can significantly improve performance.

phiresky

#!/bin/bash

fname="$1"

cachedir=/tmp/pdfextract
mkdir -p "$cachedir"

mtime="$(stat -c %Y "$1")"

hash=$(echo $fname.$mtime | sha256sum |  cut -c1-64)

phiresky

# vim: set ft=python:

# see the README at https://gist.github.com/phiresky/4bfcfbbd05b3c2ed8645
# source: https://github.com/mpv-player/mpv/issues/2149
# source: https://github.com/mpv-player/mpv/issues/566
# source: https://github.com/haasn/gentoo-conf/blob/nanodesu/home/nand/.mpv/filters/mvtools.vpy

import vapoursynth

core = vapoursynth.get_core()

phiresky

function calc(base) {
  function retryDelaySeconds(retry) {
    return Math.pow(base, retry);
  }

  function timeToStr(sec) {
    sec = Math.round(sec);
    if (sec < 60) {
      return sec + "s";
    }

phiresky

set -e
set -x

src=backup
dst=borg
tmpdir=./tmp

t1=$(mktemp)

rdiff-backup --list-increments $src > $t1

phiresky

// reorders amazon search results to sort by price per weight to find the actually cheapest item
// 1. install https://addons.mozilla.org/en-US/firefox/addon/weautopagerize/
// 2. search for something
// 3. scroll down all the way
// 4. run this script

// "featured by amazon"
for (let crap of document.getElementsByClassName(
    "template=FEATURED_ASINS_LIST"
))

phiresky

tiny guitar synth in 96 chars of C

works by starting with a array filled with white noise (from /dev/urandom), then continuously modulating it with a low pass filter of the desired frequency.

this results in a sound pretty similar to a guitar with steel or nylon strings.

phiresky

Using SQL databases in a typed language is a pain unless you have great libraries to support you. There's a lot of different libraries for TypeScript, but they all have flaws.

This is complete overview of SQL libraries for TypeScript. If I'm missing a library, please let me know.

Object Relation Mappers (ORMs)

In an ORM you declare the schema completely in the host language (TypeScript). The ORM then completely manages synchronization between your objects / classes and the corresponding database tables.

ORMs always have the same issues: If you have somewhat complex queries, you will get to the limit of the ORM and not be able to represent that query in it without escape hatching. You also lose direct control over how the queries are handled, and thus may get surprising performance issues when the ORM uses dumb SQL queries in the background.

phiresky

// cjk tokenization snippet:

"hello test 德国 をクリックしてください 안녕하세요 세계"
    .replace(/(\p{Script=Han}|\p{Script=Hiragana}|\p{Script=Katakana}|\p{Script=Hang})/ug, "$1 ")

// inserts spaces after every CJK character, so a normal tokenizer will pick up each character as
// a separate "word". there doesn't seem to be a smarter alternative to this.