Skip to content

Instantly share code, notes, and snippets.

@drsnyder
drsnyder / urandom-reads.py
Last active August 11, 2021 10:44
Demonstrate the contention on /dev/urandom with N threads.
# Create a user land file for testing.
# dd if=/dev/urandom of=/tmp/urandom bs=1M count=10
#
# urandom-reads.py infile threads
# Examples:
# time python2.6 urandom-reads.py /tmp/urandom
# time python2.6 urandom-reads.py /dev/urandom
#
# R to generate a plot of the read time distribution at each level of concurrency
# rdt = read.csv("output.csv", header=F)
@drsnyder
drsnyder / boltzmann.R
Created March 4, 2013 05:46
Gibbs or Boltzmann distribution plot in R.
boltzmann <- function(x, t=0.1) { exp(x/t) / sum(exp(x/t)) }
x=rnorm(10,mean=1,sd=0.5)
plot(boltzmann(x))
lines(boltzmann(x))
@drsnyder
drsnyder / deadlock-postgresql-upsert.sh
Last active May 11, 2019 18:04
Deadlock postgresql upsert.
#!/bin/bash
#CREATE TABLE poky (
#bucket varchar(256) NOT NULL,
#key varchar(1024) NOT NULL,
#data text,
#created_at timestamptz NOT NULL DEFAULT NOW() CONSTRAINT created_at_utc_check CHECK (EXTRACT(TIMEZONE FROM created_at) = '0'),
#modified_at timestamptz NOT NULL DEFAULT NOW() CONSTRAINT modified_at_utc_ch
@drsnyder
drsnyder / qubole.py
Last active September 12, 2016 19:54
Qubole to CSV
__author__ = 'minesh'
from optparse import OptionParser
from qds_sdk.commands import *
from io import *
class ResultsFP:
def __init__(self, filename):
self.filename = filename
def write(self, res):
@drsnyder
drsnyder / trending-threads.json
Created June 16, 2016 16:08
Trending threads JSON payload
@drsnyder
drsnyder / perf-stats.sh
Created January 27, 2014 19:21
Simple perf stats
for i in {1..20}; do ( time -p curl -I -s http://www.site.com ) 2>&1 | grep real | cut -c6-; done | sort -n | awk '{ d[NR]=$1; } END { print NR, "-", d[1], d[int(NR*0.5)], d[int(NR*0.95)], d[NR]; }'
@drsnyder
drsnyder / counters-explain-withpk.txt
Created January 3, 2014 20:24
EXPLAIN ANALYZE on counters WITH primary key on (count_type, count_id)
test=# explain (analyze, buffers, verbose) update counters set count = count + 1 where count_type = 0 and count_id = 1;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------
Update on public.counters (cost=0.00..5.03 rows=1 width=18) (actual time=0.036..0.036 rows=0 loops=1)
Buffers: shared hit=6
-> Seq Scan on public.counters (cost=0.00..5.03 rows=1 width=18) (actual time=0.020..0.021 rows=1 loops=1)
Output: count_type, count_id, (count + 1), ctid
Filter: ((counters.count_type = 0) AND (counters.count_id = 1))
Rows Removed by Filter: 4
Buffers: shared hit=5
@drsnyder
drsnyder / counters-explain-npk.txt
Created January 3, 2014 20:09
EXPLAIN ANALYZE on counters with no primary or unique key.
test=# explain (analyze, buffers, verbose) update counters set count = count + 1 where count_type = 0 and count_id = 1;
QUERY PLAN ------------------------------------------------------------------------------------------------------------------
Update on public.counters (cost=0.00..5.74 rows=15 width=18) (actual time=0.110..0.110 rows=0 loops=1)
Buffers: shared hit=20
-> Seq Scan on public.counters (cost=0.00..5.74 rows=15 width=18) (actual time=0.014..0.054 rows=15 loops=1)
Output: count_type, count_id, (count + 1), ctid
Filter: ((counters.count_type = 0) AND (counters.count_id = 1))
Rows Removed by Filter: 32
Buffers: shared hit=5
Total runtime: 0.137 ms
CREATE TABLE counters (
count_type INTEGER NOT NULL,
count_id INTEGER NOT NULL,
count INTEGER NOT NULL
);
-- later to fix the issue
ALTER TABLE counters ADD PRIMARY KEY (count_type, count_id);
CREATE TABLE primary_relation (
@drsnyder
drsnyder / counters.py
Created January 2, 2014 16:40
A simplified counters example with a deadlock issue. The SQL is here https://gist.github.com/drsnyder/8222047.
import os
import random
import time
import psycopg2
COUNTERS = 5
THREADS = 10
ITERATIONS = 500
def increment():