Skip to content

Instantly share code, notes, and snippets.

@guyboertje
guyboertje / log-sample.xml
Created Jan 31, 2019
Logstash xml file processing
View log-sample.xml
<?xml version="1.0" encoding="UTF-8" ?>
<ExecutionLogModel>
<LogEvent level="INFO" time="2018-07-10T04:12:26Z" shapename="initializing..." shapetype="initializing..." shapelabel="" shapeextendedinfo="">
<Message>Executing Process Performance Management- Past(Child)-STAGE (Continuation f_0_0)</Message>
</LogEvent>
<LogEvent level="INFO" time="2018-07-10T04:12:26Z" shapename="shape10" shapetype="Try/Catch" shapelabel="" shapeextendedinfo="">
<Message>Executing Try/Catch Shape continuation as f_0_0 with 1 document(s).</Message>
</LogEvent>
<LogEvent level="INFO" time="2018-07-10T04:12:26Z" shapename="shape10" shapetype="Try/Catch" shapelabel="" shapeextendedinfo="">
<Message>Shape executed successfully in 82 ms.</Message>
@guyboertje
guyboertje / queue.rake
Created Dec 1, 2016
rakelib/queue.rake
View queue.rake
namespace "queue" do
desc "print out contents of checkpoint file (specify path)"
task "print-checkpoint", [:ckpfile] do |t, args|
bin = IO.read(args.ckpfile, mode: "rb")
# + Short.BYTES // version 16 bits
# + Integer.BYTES // pageNum 32 bits
# + Integer.BYTES // firstUnackedPageNum 32 bits
# + Long.BYTES // firstUnackedSeqNum 64 bits
# + Long.BYTES // minSeqNum 64 bits
# + Integer.BYTES // eventCount 32 bits
View timeout_with_threads.rb
def enqueue(event)
q1 = SizedQueue.new(1)
tt = Thread.new(q1) do |q|
sleep 5
q.push 42
end
wt = Thread.new(q1) do |q|
@otherqueue.push event
View original_translate_spec.rb
# encoding: utf-8
require "logstash/devutils/rspec/spec_helper"
require "logstash/filters/translate"
describe LogStash::Filters::Translate do
let(:config) { Hash.new }
subject { described_class.new(config) }
describe "exact translation" do
@guyboertje
guyboertje / irb_output.txt
Created Sep 16, 2015
Check of sleeping threads
View irb_output.txt
irb(main):067:0> s2 = Supervisor2.new
=> #<Supervisor2:0x5fbdfdcf>
irb(main):068:0> s2.run
sleeping for 30
=> [#<Thread:0x156b88f5 run>, #<Thread:0x3bf9ce3e run>]
irb(main):069:0> s2.stop
=> [#<Thread:0x156b88f5 dead>, #<Thread:0x3bf9ce3e run>]
irb(main):070:0> s3 = Supervisor3.new
=> #<Supervisor3:0x71def8f8>
irb(main):071:0> s3.run
@guyboertje
guyboertje / cabin_helper.rb
Created Aug 19, 2015
cabin mocking version 2
View cabin_helper.rb
class LogTracker
def filter(&block)
cache.detect(&block)
end
def cache()
@cache ||= []
end
def <<(hash)
@guyboertje
guyboertje / cabin_mocking.rb
Last active Aug 29, 2015
Cabin mocking - it can be done
View cabin_mocking.rb
class LogTracker
def cache()
@cache ||= Hash.new{|h,k| h[k] = []}
end
def <<(hash)
hash.each{|k,v| cache[k].push(v) }
end
end
@guyboertje
guyboertje / logstash-pipe-bench.txt
Last active Aug 29, 2015
Logstash pipeline bench report
View logstash-pipe-bench.txt
For all below, this applies...
ruby -v
jruby 1.7.20 (1.9.3p551) 2015-05-04 3086e6a on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [darwin-x86_64]
1) As a baseline for experimentation I substitute the JSON parser and generator for 'perfect' simulations.
So for parse I simply return a new Hash and for generate I return a new String with 1 interpolation.
yes '{"test": 1}' | bin/logstash -e 'input{stdin{codec => json_lines}} filter{clone{}} output{stdout{codec => json_lines}}' | pv -Wlart > /dev/null
01:00 [89.3k/s] [ 89k/s]
View Ruby.tmLanguage
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>comment</key>
<string>
TODO: unresolved issues
text:
"p &lt;&lt; end
@guyboertje
guyboertje / some_model.rb
Last active Aug 29, 2015
Streaming Json parsing: implest class to build from Oj.saj_parse and JrJackson::Json.sj_parse
View some_model.rb
class SomeModel
def null(*) end
ATTRS = ['v_string', 'v_num', 'v_boolean', 'v_decimal']
attr_accessor *ATTRS.map(&:to_sym)
METHODMAP = Hash.new(instance_method(:null))
ATTRS.each {|attr| METHODMAP[attr] = instance_method("#{attr}=") }
def self.load(hash) new.load(hash); end
def load(hash)