Skip to content

Instantly share code, notes, and snippets.

# Sidekiq Reporter
#
# Emit key statistics about Sidekiq queues to a stream.
#
# Examples:
#
# Log to STDOUT by default:
#
# ruby sidekiq_reporter.rb
#
# ~/.bash_profile
if [ -f `brew --prefix`/etc/bash_completion ]; then
. `brew --prefix`/etc/bash_completion
fi
if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi
GIT_PS1_SHOWDIRTYSTATE=true
source ~/.git-prompt.sh
input {
  # translate syslog messages into logstash events
  # with priority field, fields added by SYSLOGLINE pattern
  # (e.g. timestamp, logsource, program, pid, etc.) and the 
  # rest of the syslog string in the message field
  syslog {
    # port => 514
    # codec => plain
    # syslog_field => "message"
require 'openssl'
require 'base64'
cipher = OpenSSL::Cipher::Cipher.new("aes-256-cbc")
key = cipher.random_key
value = Base64.strict_encode64(key)
irb
allocation = {
events: 3_131_647.7,
fanout: 1_234_271.5,
default: 17_179_872,
detonation: 705_552.8,
require 'openssl'
require 'base64'
module Cipher
extend self
def encrypt(plaintext)
encode(aes(plaintext))
end

RF=3, Ring=3, Up=3

write

  Execute CQL3 query | 2017-12-15 09:30:45.114000 | 127.0.0.1 |              0
            Parsing update count_by_id SET count = count + 1 WHERE id = 1; [SharedPool-Worker-1] | 2017-12-15 09:30:45.114000 | 127.0.0.1 |            320
                                                       Preparing statement [SharedPool-Worker-1] | 2017-12-15 09:30:45.120000 | 127.0.0.1 |           5772
                                 Executing single-partition query on local [SharedPool-Worker-1] | 2017-12-15 09:30:45.172000 | 127.0.0.1 |          57252
                                              Acquiring sstable references [SharedPool-Worker-1] | 2017-12-15 09:30:45.172000 | 127.0.0.1 |          57735

MRI GIL isn't pure evil, a.k.a. Native Threads Aren't Magic

The GIL in MRI does not inherintly mean inefficient concurrency. Likewise, using "native threads" does not inherintly mean efficient concurrency.

To be clear, modern ruby (>=1.9) does use native threads. The key is that GIL locking ensures that only 1 thread is executing at a time (regardless of the number of CPU cores available), per process. So, MRI threads are native threads, but to draw the distinction, we'll use the terms "GIL thread" and "native thread" below.

Scheduling

Concurrency is not about magically getting two threads to execute at the same time on a single core (because that would be magic). It is about efficiently managing thread scheduling so the maximum amount of CPU cycles is spent doing work.

### Stub for a logstash filter event
class Event
def initialize(initial_data=nil)
@data = initial_data || {}
end
def get(key)
@data[key]
end