Skip to content

Instantly share code, notes, and snippets.

@8enmann
8enmann / .zshrc
Created January 19, 2021 00:26
dotfiles
# Enable Powerlevel10k instant prompt. Should stay close to the top of ~/.zshrc.
# Initialization code that may require console input (password prompts, [y/n]
# confirmations, etc.) must go above this block; everything else may go below.
if [[ -r "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh" ]]; then
source "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh"
fi
# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH
@8enmann
8enmann / test_process.py
Created June 24, 2019 20:46
Partly broken example trying to test Process as a single thread.
# mock_process.py
"""Make a process single threaded for testing."""
import multiprocessing
def f(x):
return x
def target():
"""Run a process by specifying a target function."""
@8enmann
8enmann / barrier_example.py
Created June 24, 2019 18:10
Simple Barrier demo
# barrier_example.py
"""Example usage of barrier."""
import multiprocessing as mp
import itertools
def worker(barrier:mp.Barrier, q:mp.Queue=None):
print(mp.current_process().name, 'waiting')
i = barrier.wait()
if i == 0:
@8enmann
8enmann / bpe.py
Created June 24, 2019 03:29
Approximate BPE implementation.
"""Implements an approximate BPE encoding over bytes with some tricks for efficiency.
https://arxiv.org/pdf/1508.07909.pdf section 3.2.
Basic algorithm from the paper:
Initialize the vocab with the character vocabulary
Each word is a sequence of characters plus an enod of word symbol '·'
Count all symbol pairs
Replace each occurence of the most frequent pair ('a', 'b') with 'ab'.
Each merge represents a character n-gram
@8enmann
8enmann / binary_search.py
Created June 23, 2019 19:17
Practice implementation of binary search.
"""Practice implementation of binary search."""
import pytest
from typing import Sequence
def binary_search(sorted_arr: Sequence[int], target:int) -> int:
"""Find the index of target in sorted_arr.
Returns:
Index of the target or -1 if not found. Picks the first match randomly if there are multiple.
@8enmann
8enmann / ngram.py
Created June 22, 2019 21:03
Calculate ngrams as fast as possible
"""Calculate N-gram counts as fast as possible for a large encoded file."""
import numpy as np
from collections import deque
import multiprocessing as mp
from typing import Any, Iterable, Generator
import time
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
from functools import partial
@8enmann
8enmann / counter.py
Created June 22, 2019 20:18
Experimenting with pytest, cosine similarity, ngram counting
import requests
import re
from typing import List, Sequence
from collections import Counter
import numpy as np
TEST_URL = 'http://titan.dcs.bbk.ac.uk/~kikpef01/testpage.html'
URL_RE = re.compile(r'href="(http.+?)"')
TOKENIZER_RE: re.Pattern = re.compile(r'\w+')
@8enmann
8enmann / count.py
Last active June 22, 2019 20:14
Benchmark reading files with threading and multiprocessing
"""Read files using threading and multiprocessing.
Execute on https://coderpad.io/sandbox
"""
from concurrent.futures import ThreadPoolExecutor, as_completed
from multiprocessing.dummy import Pool
import multiprocessing
from collections import Counter
import glob
@8enmann
8enmann / test_format_time.py
Created June 21, 2019 02:05
Format time in Python using pytest
"""
TDD solution to:
Input: a bunch of times, minutes and seconds, formatted as a single string like: "12:32 34:01 15:23 9:27 55:22 25:56"
Output: the sum of the times, hours, minutes, and seconds, formatted as a single string like: "2:32:41"
https://github.com/cjdev/interview-preparation
https://coderpad.io/sandbox
"""
import pytest
======================================== SAMPLE 1 ========================================
breakfast," the janitor said. "I'm assuming it's an abundance day so everyone has extra for one coming to go," he added. The cream and butter was nice and fluffy, though I
======================================== SAMPLE 2 ========================================
French recipe before I went to college so I've learned a lot here. I'm not going to spend too much time about it, but for those of you who don't, it's a simple
======================================== SAMPLE 3 ========================================
meal from New Order for the last few weeks, and don't think that I can take days off."
Little did she know, the waiter was among the first to take advantage of both the
======================================== SAMPLE 4 ========================================
white pepper cake with my boyfriend.