Skip to content

Instantly share code, notes, and snippets.

David Liman dvliman

View GitHub Profile
View go env
go env
GOARCH="amd64"
GOBIN="/usr/local/go/bin"
GOCACHE="/Users/dv/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/dv/go"
View concurrency-models
concurrency-models
View keybase.md

Keybase proof

I hereby claim:

  • I am dvliman on github.
  • I am dvliman (https://keybase.io/dvliman) on keybase.
  • I have a public key ASBe4DkFe9qsB1j_Dc89k_Vd2euOfMD-Qz4FXTtbNRwTtwo

To claim this, I am signing this object:

@dvliman
dvliman / spike-day-02-03-20.md
Created Feb 9, 2020 — forked from cldwalker/spike-day-02-03-20.md
GraalVM dive in Clojure at work
View spike-day-02-03-20.md

Spike

I looked into the state of GraalVM and Clojure and wrote some small work-related scripts.

GraalVM Build Tools

@dvliman
dvliman / gist:921ca48137d134e0152d494d9aa54c73
Created Nov 14, 2019 — forked from chanks/gist:7585810
Turning PostgreSQL into a queue serving 10,000 jobs per second
View gist:921ca48137d134e0152d494d9aa54c73

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

View dummy.go
package httplib
import (
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/manyminds/api2go"
View config.fish
set -gx PATH $PATH /usr/local/sbin
set -gx GOPATH $HOME/go
set -gx GOBIN $GOPATH/bin
set -gx PATH $PATH $GOBIN
set -gx GOOGLE_APPLICATION_CREDENTIALS /Users/dliman/lereta/acq-plt/app-credentials.json
set -gx GOGOOGLE_APPLICATION_CREDENTIALS /Users/dliman/lereta/acq-plt/app-credentials.json
set -gx APP_ENV local
set -gx PROJECT_ID ltf-eng-dliman-cb8b
function set_psql
View rename-files
# rename files to have underscores instead of ' ', '(', and ')' - runs on * or takes extension argument: i.e., `rename_filename_chars txt`
rename_filename_chars() { for file in *"$1"; do mv -n "$file" $(echo $file | sed -e "s/ /_/g" | sed -e "s/(/_/g" | sed -e "s/)/_/g"); done }
You can’t perform that action at this time.