Skip to content

Instantly share code, notes, and snippets.

Avatar

Martin Thurau martinth

View GitHub Profile
@ursuad
ursuad / kafka-cheat-sheet.md
Last active Oct 26, 2021
Quick command reference for Apache Kafka
View kafka-cheat-sheet.md

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

View pyvideo_popularity.py
#!/usr/bin/env python3
# encoding: utf-8
"""
I saw a similar script on the homepage of Miguel Grinberg (the Flask book guy),
but he was using webscraping. Here I use simple API calls instead.
The script takes the presentations of a Python conference and orders the
presentations in descending order by the number of youtube views. It
is an indicator about the popularity of a video.
@acusti
acusti / google-analytics.js
Last active Nov 7, 2017 — forked from ismyrnow/google-analytics-amd.js
Barebones google analytics RequireJS module wrapper for “universal analytics” with example of how to use it within a single page application
View google-analytics.js
/**
* Google analytics include (using "Universal Analytics")
* https://gist.github.com/acusti/8718758
*/
/*global define */
define(function(require) {
'use strict';
@chanks
chanks / gist:7585810
Last active Oct 25, 2021
Turning PostgreSQL into a queue serving 10,000 jobs per second
View gist:7585810

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@ebouchut
ebouchut / WhereDoesMavenDependencyComesFrom
Last active Nov 25, 2020
How to check where a maven dependency comes from #maven #find #dependency
View WhereDoesMavenDependencyComesFrom
#
# To check from where a maven dependency comes from
#
mvn dependency:tree -Dincludes=groupId:artifactId