Skip to content

Instantly share code, notes, and snippets.

# memory_pool.pyx
from libc.stdlib cimport malloc, free
from libc.string cimport memcpy
from cpython.bytes cimport PyBytes_AsString, PyBytes_FromStringAndSize
from threading import Lock
from orso.tools import random_int
from libcpp.vector cimport vector
cdef struct MemorySegment:
int start
@joocer
joocer / lru_k.py
Created September 14, 2022 14:05
LRU-K implementation in Python
"""
LRU-K evicts the page whose K-th most recent access is furthest in the past.
This is a basic implementation of LRU-2, which evicts entries according to the time
of their penultimate access. The main benefit of this approach is to prevent
a problem when the items being checked exceeds the number of items in the cache. A
classic LRU will evict and repopulate the cache for every call. LRU-2 reduces the
likelihood of this, but not preferring the MRU item to be retained.
LRU-K should be used in conjunction with eviction limits per query - this appears to
@joocer
joocer / color-gradient.js
Last active January 26, 2022 17:43 — forked from gskema/color-gradient.js
Generate gradient color from an arbitrary number of colors
function colorGradient(colors, fadeFraction) {
if (fadeFraction >= 1) {
return colors[colors.length - 1]
} else if (fadeFraction <= 0) {
return colors[0]
}
let fade = fadeFraction * (colors.length - 1);
let interval = Math.trunc(fade);
fade = fade - interval
@joocer
joocer / boolparser.py
Last active August 2, 2021 13:40 — forked from leehsueh/boolparser.py
Python Boolean Expression Parser/Evaluator
"""
Grammar:
========
Expression --> AndTerm { OR AndTerm}+
AndTerm --> Condition { AND Condition}+
Condition --> Terminal (>,<,>=,<=,==) Terminal | (Expression)
Terminal --> Number or String or Variable
Usage:
======
@joocer
joocer / bollinger.py
Created June 15, 2021 20:35
python bollinger bands without pandas
from matplotlib import pyplot as plt # type:ignore
from mabel.data.formats import dictset
import numpy as np # type:ignore
def bollinger_bands(series, length=20, sigmas=2):
index = 0
window = series[0:length]
num_vals = len(window)
@joocer
joocer / histogram
Created March 8, 2021 20:37
single char high histogram
bar_chars = (' ', '▁', '▂', '▃', '▄', '▅', '▆', '▇', '█')
def _draw_histogram(bins):
mx = max([v for k,v in bins.items()])
bar_height = (mx / 7)
if mx == 0:
return ' ' * len(bins)
histogram = ''
for k,v in bins.items():
@joocer
joocer / bplustree.py
Last active April 25, 2021 19:34 — forked from savarin/bplustree.py
Python implementation of a B+ tree
"""
B+Tree Code Adapted From:
https://gist.github.com/savarin/69acd246302567395f65ad6b97ee503d
No explicit license when accessed on 2nd March 2020.
Other code:
(C) 2021 Justin Joyce.
@joocer
joocer / gcs_threaded_reader.py
Created October 24, 2020 09:46
Reads a set of blobs parallel to speed up reading. Blobs are read line by line, the usecase this was written for was jsonl files, but any record-per-line format where you don't care about the order of the lines should not have issues.
"""
Threaded Google Cloud Storage Blob reader.
Reads a set of blobs parallel to speed up reading. Blobs are
read line by line, the usecase this was written for was jsonl
files, but any record-per-line format where you don't care
about the order of the lines should not have issues.
Limited performance testing reading a set of eight blobs,
4x 19Mb, 4x 5.4Mb in four threads ran in 20.1% of the time.
@joocer
joocer / gcs_handlers.py
Last active October 22, 2020 14:18
simplify the handling of GCS blobs in Python
import datetime, re
from google.cloud import storage
from functools import lru_cache
try:
import ujson as json
except ImportError:
import json
def select_dictionary_values(dictionary, values):
@joocer
joocer / read_large_file.py
Last active October 24, 2020 09:48
read an arbitrary long file in python, line by line
def read_file(filename, chunk_size=1024*1024, delimiter='\n'):
with open(filename, 'r', encoding="utf8") as f:
carry_forward = ''
chunk = 'INITIALIZED'
while len(chunk) > 0:
chunk = f.read(chunk_size)
augmented_chunk = carry_forward + chunk
lines = augmented_chunk.split(delimiter)
carry_forward = lines.pop()
yield from lines