Skip to content

Instantly share code, notes, and snippets.

@sevagh
sevagh / fs_trie.rs
Created November 30, 2019 05:38
Rust trie that serializes to filesystem
//! # A trie that can be saved to and loaded from a file
//!
//! This crate implements a Trie with char keys.
//! The trie can be saved to and loaded from a file on the local filesystem.
//! This allows the user to persist the trie between executions.
//!
//! Basic example:
//!
//! ```ignore
//! let trie_file = "/path/to/trie-file";
@sevagh
sevagh / multiheadattention.cpp
Created November 28, 2023 18:06
multihead attention in Eigen/C++
void demucscpp::common_encoder_layer(
Eigen::Tensor3dXf &q, // q = x = frequency
const Eigen::Tensor3dXf &k, // k = xt = time
const Eigen::Tensor1dXf &norm1_weight, const Eigen::Tensor1dXf &norm1_bias,
const Eigen::Tensor1dXf &norm2_weight, const Eigen::Tensor1dXf &norm2_bias,
const Eigen::MatrixXf &in_proj_weight, const Eigen::VectorXf &in_proj_bias,
const Eigen::MatrixXf &out_proj_weight,
const Eigen::VectorXf &out_proj_bias, const Eigen::VectorXf &gamma1_scale,
const Eigen::Tensor1dXf &norm3_weight, const Eigen::Tensor1dXf &norm3_bias,
const Eigen::MatrixXf &linear1_weight, const Eigen::VectorXf &linear1_bias,
@sevagh
sevagh / eigen_layer_norm
Created November 21, 2023 12:46
my naive (and incorrect) code for computing torch.nn.LayerNorm with Eigen3
Eigen::Tensor3dXf demucscpp::layer_norm_last_dim(const Eigen::Tensor3dXf &x,
const Eigen::Tensor1dXf &weight,
const Eigen::Tensor1dXf &bias,
const float eps=1e-5) {
// compute layer norm across last dimension
// eps is typically 1e-5
//
// e.g. x = (A, B, C)
// w = (C)
// b = (C)
@sevagh
sevagh / bootstrap-mdx-cpp.sh
Last active August 13, 2023 14:07
create newproj.cpp (oriented for music demixing projects e.g. demucs.cpp)
#!/usr/bin/env bash
# script for bootstrapping an '$mdx.cpp' project
# mdx = music demixing
# e.g. umx.cpp (open-unmix)
# demucs.cpp (Demucs)
set -Eou pipefail
set -x
@sevagh
sevagh / periphery2musdb.py
Last active August 26, 2022 13:09
convert periphery stems to musdb
import os
import numpy
import itertools
import soundfile
import sys
import multiprocessing
import argparse
sample_rate = 44100
@sevagh
sevagh / cupy_oom_problem.py
Created April 22, 2021 15:33
demo of cupy out of memory issue with large ndarray
import numpy as np
import scipy.fft
import cupyx
import cupy
# managed memory to stream large ndarrays to cuda
cupy.cuda.set_allocator(cupy.cuda.MemoryPool(cupy.cuda.memory.malloc_managed).malloc)
@sevagh
sevagh / bloom_filter.rs
Created January 7, 2020 03:28
basic Bloom Filter with 4 hashing functions
use bit_vec::BitVec;
use std::hash::{Hash, Hasher};
// use 4 hash functions
use ahash::AHasher;
use fnv::FnvHasher;
use metrohash::MetroHash64;
use siphasher::sip::SipHasher;
pub struct BloomFilter {
@sevagh
sevagh / quadtree_sparse_matrix.py
Created December 7, 2019 14:41
scipy-quadtree-sparse-matrix
'''
quadtree.py
A matrix implementation based on a pointer region quadtree, implemented in pure Python.
The script takes a scipy sparse matrix format as the only argument and calls 'eval' to use that format as the control matrix.
See: https://docs.scipy.org/doc/scipy/reference/sparse.html
Several sizes and densities are tested (all matrices are square). Measured:
@sevagh
sevagh / pppid.h
Created November 30, 2019 05:49
Get pid at any depth with procfs
#include <sys/types.h>
#include <algorithm>
#include <cctype>
#include <fstream>
#include <string>
#define PROC_STR "/proc/%s/stat"
pid_t
_getpppid(const char *, int);
@sevagh
sevagh / palette.py
Created November 30, 2019 05:44
Count colors in an image
#!/usr/bin/env python3
from collections import Counter
import numpy
from PIL import Image
from webcolors import css3_hex_to_names, hex_to_rgb
from urllib.request import urlopen
import sys
import time
import matplotlib.pylab as plb