Skip to content

Instantly share code, notes, and snippets.

If 2fa is enabled on github switch to ssh instead of https on linux
1. generate an ssh keypair on your linux box
ssh-keygen -t {rsa|dsa}
2. add the public key to github: profile - settings - ssh keys
3. switch from https to ssh
Check your repo remote:
@root-11
root-11 / compression_benchmark.py
Created July 20, 2022 09:36 — forked from oldcai/compression_benchmark.py
zlib vs lz4 vs lzma compression
import time
import requests
import zlib
# pip install lz4 pylzma
import lz4
import pylzma as lzma
def analysis_ratios(source):
time_before_zip = time.time()
from tqdm import tqdm
import requests
# file url
url = 'https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-stub-meta-history.xml.gz' #file size is '70.5GB'
# stream true is required
response = requests.get(url, stream=True)
# total file size
t = int(response.headers.get('content-length', 0))
block_size = 1024**2 #1 Mbit
@root-11
root-11 / latency.txt
Created February 7, 2022 11:07 — forked from jboner/latency.txt
Latency Numbers Every Programmer Should Know
Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns 3 us
Send 1K bytes over 1 Gbps network 10,000 ns 10 us
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD
@root-11
root-11 / kwarg_filter
Last active October 27, 2021 13:36
Decorator for filtering keywords from a big dict.
import functools
import inspect
def kwarg_filter():
def decorator(fn):
keys = inspect.getfullargspec(fn).args # <-- This reads the signature of the function.
@functools.wraps(fn)
def inner(*args, **kwargs):
@root-11
root-11 / minimal_test_generator.py
Created April 21, 2020 10:31
A minimal test generator example to automatic creation of tests when code fails.
def adder(a,b): # function to test.
""" adds a to b"""
return a+b
options = [9, 9.0, [9], "9"] # options for test function
test_template = """
@root-11
root-11 / profiling_decorator.py
Created July 23, 2019 08:17
profiling decorator
import functools
import time
class Stack(object):
hits = {}
times = {}
@classmethod
def register(cls, func_name, call_time):
if func_name not in Stack.hits:
@root-11
root-11 / concurrent futures with FIFO
Created January 8, 2014 09:25
From Linkedin Python post "Python parallelism in one line - A Better Model for Day to Day Threading Tasks" Does anyone know of similar "compact" examples of handling parallelism for the case where the workers may produce tasks that are put into the FIFO queue (maybe with `while` instead of map) ?
# requires python 3.2.3 or higher
import concurrent.futures
def f(x):
print ("working on task: %s" %{x})
if x-1 > 0 :
return x-1 # I want this task back in the queue.
def main():
Queue = [x for x in range(4)]