Skip to content

Instantly share code, notes, and snippets.

@chanks
chanks / gist:7585810
Last active July 22, 2025 01:00
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@jibsen
jibsen / bytes.md
Last active December 20, 2023 10:02
Ramblings about uint8_t and undefined behavior

Introduction

The C standard only specifies minimum limits for the values of character types and standard integer types. This makes it possible to generate efficient code on diverse architectures, but can pose problematic if your code expects the limits to match your development platform, or if you have to do low-level things.

Before C99, the usual way to solve this was to use typedef to declare synonyms

API Design: Coroutines APIs (Janurary-2017)

I am currently dealing with a lot of libraries at work. Both third party as well as libraries written or being currently in process of being written by me. I absolutely love writing and working with libraries. Especially if they present or bring me to either a new or different approach to solve a problem. Or at least provide a different view.

Over time I noticed however that quite regulary we had to decide that we cannot use a third party library. Often it is the usual reason.

Graphical User Interfaces

Last time I wrote a little bit about my current GUI research progress. There are some things that were misunderstood so I want to clarify some fundamental design decisions and finally write up some current problems and their solutions.

First up I want to clarify a component is not another term for what is usually refered to as widget. Instead wigets in this implementation are made out of n >= 1 components. Components themself are just rectangles with attributes left, right, top, bottom, center_x, center_y, width and height some behavior flags and an optional surface to draw into. For example a scroll regions is made up out of at least three

@antirez
antirez / lmdb.tcl
Created April 28, 2017 15:40
LMDB -- First version of Redis written in Tcl
# LVDB - LLOOGG Memory DB
# Copyriht (C) 2009 Salvatore Sanfilippo <antirez@gmail.com>
# All Rights Reserved
# TODO
# - cron with cleanup of timedout clients, automatic dump
# - the dump should use array startsearch to write it line by line
# and may just use gets to read element by element and load the whole state.
# - 'help','stopserver','saveandstopserver','save','load','reset','keys' commands.
# - ttl with milliseconds resolution 'ttl a 1000'. Check ttl in dump!
@kfalconspb
kfalconspb / boxstarter.ps1
Last active April 18, 2022 17:13 — forked from ElJefeDSecurIT/boxstarter.ps1
Boxstarter Commands for a new Windows box.
# Description: Boxstarter Script
# Author: kfalconspb
# Last Updated: 20180429
#
# Install boxstarter:
# . { iwr -useb http://boxstarter.org/bootstrapper.ps1 } | iex; get-boxstarter -Force
#
# You might need to set: Set-ExecutionPolicy RemoteSigned
#
# Run this boxstarter by calling the following from an **elevated** command-prompt:
@WilliamBundy
WilliamBundy / wb_alloc.h
Created November 3, 2017 01:05
A single-file header allocation library for C (probably bugs out in C++, sorry)
/* This is free and unencumbered software released into the public domain. */
/* Check the bottom of this file for the remainder of the unlicense */
/* wb_alloc.h
*
* Three custom allocators that can (hopefully) safely allocate a very large
* amount of memory for you. Check the warnings below for an explanation
*
* Version 0.0.1 Testing Alpha
*/
@timvisee
timvisee / falsehoods-programming-time-list.md
Last active October 31, 2025 18:31
Falsehoods programmers believe about time, in a single list

Falsehoods programmers believe about time

This is a compiled list of falsehoods programmers tend to believe about working with time.

Don't re-invent a date time library yourself. If you think you understand everything about time, you're probably doing it wrong.

Falsehoods

  • There are always 24 hours in a day.
  • February is always 28 days long.
  • Any 24-hour period will always begin and end in the same day (or week, or month).
@yogthos
yogthos / clojure-beginner.md
Last active October 5, 2025 02:49
Clojure beginner resources

Introductory resources