Skip to content

Instantly share code, notes, and snippets.

View 17twenty's full-sized avatar

Nick Glynn 17twenty

View GitHub Profile
@17twenty
17twenty / gist:ca503d3be60330e3b3d07799fcdbdc33
Created November 14, 2019 19:03 — forked from chanks/gist:7585810
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@17twenty
17twenty / 1_kubernetes_on_macOS.md
Created June 18, 2018 03:45 — forked from kevin-smets/1_kubernetes_on_macOS.md
Local Kubernetes setup on macOS with minikube on VirtualBox and local Docker registry

Requirements

Minikube requires that VT-x/AMD-v virtualization is enabled in BIOS. To check that this is enabled on OSX / macOS run:

sysctl -a | grep machdep.cpu.features | grep VMX

If there's output, you're good!

Prerequisites

@17twenty
17twenty / le-cert.rb
Created March 23, 2017 10:32 — forked from 0x7466/le-cert.rb
Easy to use Ruby script to register and authorize new domains on Let's Encrypt and getting certificates for them. Call 'ruby le-new.rb' and follow the instructions. Install OpenSSL and the ACMEClient gem before!
#!/usr/bin/env ruby
begin
require 'acme-client'
rescue LoadError
abort 'MISSING GEM! You haven\'t installed the ACME client. Install the gem with the command \'gem install acme-client\'.'
end
require 'openssl'
require 'resolv'
@17twenty
17twenty / drop_encrypt.go
Created March 13, 2017 22:28 — forked from josephspurrier/drop_encrypt.go
Golang - Drag and Drop AES Encryption and Decryption
package main
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"fmt"
"io"
"io/ioutil"
"os"
@17twenty
17twenty / dontLeak.go
Created November 15, 2016 05:36 — forked from rakyll/gist:1aa860377dab8fd445431bbb3204f600
How to Not Leak Go Routines using Contexts
package main
import (
"context"
"fmt"
)
func gen(ctx context.Context) <-chan int {
ch := make(chan int)
go func() {
@17twenty
17twenty / win32.go
Last active June 23, 2016 23:19 — forked from nathan-osman/win32.go
Simple Windows GUI application written in Go
package main
import (
"log"
"syscall"
"unsafe"
)
var (
kernel32 = syscall.NewLazyDLL("kernel32.dll")
@17twenty
17twenty / reader.go
Created March 10, 2016 03:22 — forked from jedy/reader.go
processes communicate with shared memory in golang
package main
// #include <stdlib.h>
// #include "wrapper.c"
import "C"
import "unsafe"
import "fmt"
func read(filename string) string {
f := C.CString(filename)
<!DOCTYPE html>
<html>
<head>
<script src="//ajax.googleapis.com/ajax/libs/angularjs/1.3.4/angular.min.js"></script>
<script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<script src="main.js"></script>
</head>
<body ng-app="myApp" ng-controller="myController">
<div>
<input type="text" ng-model="username" />
@17twenty
17twenty / _.md
Last active August 29, 2015 14:17 — forked from klange/_.md

Since this is on Hacker News and reddit...

  • No, I don't distribute my résumé like this. A friend of mine made a joke about me being the kind of person who would do this, so I did (the link on that page was added later). My actual résumé is written in BSD mandoc.
  • I apologize for the use of _t in my types. I spend a lot of time at a level where I can do that; "reserved for system libraries? I am the system libraries".
  • Since people kept complaining, I've fixed the assignments of string literals to non-const char *s.
  • My use of type * name, however, is entirely intentional.
  • If you're using an older compiler, you might have trouble with the anonymous unions and the designated initializers - I think gcc 4.4 requires some extra braces to get them working together. Anything reasonably recent should work fine. Clang and gcc (newer than 4.4, at
package com.scottyab.encryption;
/*
* This software is provided 'as-is', without any express or implied
* warranty. In no event will Google be held liable for any damages
* arising from the use of this software.
*
* Permission is granted to anyone to use this software for any purpose,
* including commercial applications, and to alter it and redistribute it
* freely, as long as the origin is not misrepresented.