enixdark

enixdark

How to config rabbitmq server cluster [3 nodes]

Edit /etc/hosts

vi /etc/hosts
192.168.10.157  rabbitmq-1
192.168.10.159  rabbitmq-2
192.168.10.161  rabbitmq-3

enixdark

# Alert when a login event is detected for user "admin" never before seen IP
# In this example, "login" logs contain which user has logged in from what IP

# (Optional)
# Elasticsearch host
es_host: localhost

# (Optional)
# Elasticsearch port
es_port: 9200

enixdark

##################################################################
# /etc/elasticsearch/elasticsearch.yml
#
# Base configuration for a write heavy cluster
#

# Cluster / Node Basics
cluster.name: logng

# Node can have abritrary attributes we can use for routing

enixdark

from time import sleep
import signal
import sys


def on_stop_handler(signum, frame):
    print 'Exiting application...'
    sys.exit(0)

enixdark

http://www.oreilly.com/data/free/files/2014-data-science-salary-survey.pdf
http://www.oreilly.com/data/free/files/2015-data-science-salary-survey.pdf
http://www.oreilly.com/data/free/files/Data_Analytics_in_Sports.pdf
http://www.oreilly.com/data/free/files/advancing-procurement-analytics.pdf
http://www.oreilly.com/data/free/files/ai-and-medicine.pdf
http://www.oreilly.com/data/free/files/analyzing-data-in-the-internet-of-things.pdf
http://www.oreilly.com/data/free/files/analyzing-the-analyzers.pdf
http://www.oreilly.com/data/free/files/architecting-data-lakes.pdf
http://www.oreilly.com/data/free/files/being-a-data-skeptic.pdf
http://www.oreilly.com/data/free/files/big-data-analytics-emerging-architecture.pdf

enixdark

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

enixdark

References from my talk 'Without Resilience, Nothing Else Matters'

• Drift into Failure
• How Complex Systems Fail
• Antifragile: Things That Gain from Disorder
• Leverage Points: Places to Intervene in a System
• Going Solid: A Model of System Dynamics and Consequences for Patient Safety
• Resilience in Complex Adaptive Systems: Operating at the Edge of Failure
• Puppies! Now that I’ve got your attention, Complexity Theory
• [Towards Resilient Architectures: Biology

enixdark

Managing SSH authentication with Vault

Managing SSH keys with Vault requires 3 steps:

1. Setting up Vault
2. Setting up the host
3. Setting up the client / using the signed client keys

For a full documentation, see this HashiCorp Blog Post

enixdark

Transcript from Outside-In TDD

• part 1
• part 2
• part 3
• Bank Kata

Part 1