tautologicc

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

tautologicc

1. Separation of immutable and mutable logic

Quite a lot of different people have been on the same trail of thought. Gary Bernhardt's formulation of a "functional core, imperative shell" seems to be the most voiced.

"Boundaries" - Gary Bernhardt

"Imperative shell" that wraps and uses your "functional core".. The result of this is that the shell has fewer paths, but more dependencies. The core contains no dependencies, but encapsulates the different logic paths. So we’re encapsulating dependencies on one side, and business logic on the other side. Or put another way, the way to figure out the separation is by doing as much as you can without mutation, and then encapsulating the mutation separately. Functional core — Many fast unit tests. Imperative shell — Few integration tests

https://www.youtube.com/watch?v=yTkzNHF6rMs

tautologicc

• The Comprehensive Guide to Speaking at Technology Conferences in 2020 (https://www.cfpland.com/guides/speaking/)
• What I've learned after sending 147 proposals to 36 conferences in a year (https://drobinin.com/posts/what-ive-learned-after-sending-147-proposals-to-36-conferences-in-a-year/)
• Introduction to slides, a Clean Presentation Tool (https://zge.us.to/slides.html)
• A command-line based markdown presentation tool (https://github.com/visit1985/mdp)
• Giving a presentation with perfect UI/UX design (https://habr.com/en/post/471624/)
• Why Your Excellent Conference Talk Was Rejected (https://www.promptworks.com/blog/why-your-excellent-talk-was-rejected)
• Very Important Strangers (http://randsinrepose.com/archives/very-important-strangers/)
• Tips for Public Speaking (http://speaking.io)
• Presentation Skills Considered Harmful (http://seriouspony.com/blog/2013/10/4/presentation-skills-considered-harmful)
• Passionate Programmer: How to Give a Keynote (https://web.archive.org/web/20150211231805/http:/

tautologicc

.SUFFIXES:

help: Makefile ## display this help and exit
	@echo 'Usage:'
	@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $< | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "  %s\n        %s\n", $$1, $$2}'

hello: ## emit the "hello, world" message
	@echo Hello, world!

.PHONY: help compose

tautologicc

# LVDB - LLOOGG Memory DB
# Copyriht (C) 2009 Salvatore Sanfilippo <antirez@gmail.com>
# All Rights Reserved

# TODO
# - cron with cleanup of timedout clients, automatic dump
# - the dump should use array startsearch to write it line by line
#   and may just use gets to read element by element and load the whole state.
# - 'help','stopserver','saveandstopserver','save','load','reset','keys' commands.
# - ttl with milliseconds resolution 'ttl a 1000'. Check ttl in dump!

tautologicc

// SPDX-License-Identifier: CC0
extern int write();

int
main(int argc, char *argv[])
{
	if (argc < 2) {
		static char usage[] = "usage: paren STRING\n";
		write(2, usage, sizeof usage);
		return 2;

tautologicc

Q: Why don't you do open source?

A: Because I won't write useful software unless someone pays me. Also, fuck corporations — I won't work for them for free.

tautologicc

X11 forwarding on macOS and docker

A quick guide on how to setup X11 forwarding on macOS when using docker containers requiring a DISPLAY.

This guide was tested on:

• macOS Catalina 10.15.4
• docker desktop 2.2.0.5 (43884) - stable release
• XQuartz 2.7.11 (xorg-server 1.18.4)

Step-By-Step Guide

tautologicc

if [ "$(uname)" = Darwin ]; then
	alias ssh-add='ssh-add --apple-use-keychain'	
fi

export SSH_ENV="$HOME/.ssh/env"

if [ -e "$SSH_ENV" ]; then
	. "$SSH_ENV"
fi

tautologicc

1. A listening socket blocked on accept() can be unblocked by calling shutdown(socket_fd, SHUT_RD) on Linux. On Mac, calling shutdown(socket_fd, SHUT_RD) on a listening socket that is blocked on accept() will return an error ENOTCONN. To unblock a listening socket blocked on accept() on Mac requires calling close(socket_fd), which will cause the blocking accept() call to return an error EBADF.

Therefore, in order to unblock a socket on accept() on both Mac/Linux, shutdown() should not be relied on. Rather, an async cancellation scheme (such as cancellation tokens in .NET) should be used instead to unblock the call to accept().

2. When a socket is instantiated and subsequently registered to epoll, or when a socket is shut down (via. shutdown() or setsockopt(SO_LINGER)) on Linux, epoll will be notified with EPOLLHUP on the socket. On Mac, kqueue will only notify when a socket is shut down (via. shutdown() or setsockopt(SO_LINGER)) by sending a EV_EOF notification on filters EVFILT_READ, and EVFILT_WRITE.

Thi