Skip to content

Instantly share code, notes, and snippets.

@NigelThorne
Last active December 20, 2016 01:31
Show Gist options
  • Save NigelThorne/5742ba1c753df9c5b09a9f28d42eb959 to your computer and use it in GitHub Desktop.
Save NigelThorne/5742ba1c753df9c5b09a9f28d42eb959 to your computer and use it in GitHub Desktop.
Scan log files and track number of subscribers over time. (for dumping into excel)
# Run through all zip files to get log files
# Process log files => events
# Process events => (event + initial_state) => new_state => to state stream
#
require 'rubygems'
require 'zip'
require 'date'
$TIMESTAMP = "(?:^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3})"
def log_to_events(buffer, &block)
buffer.scan(/#{$TIMESTAMP}(?:(?!#{$TIMESTAMP}).)+/m, &block)
end
def update(state, event) # => new_state
return add_subscriber(state, event) if ev_is_subscription(event)
return remove_subscriber(state, event) if ev_is_unsubscription(event)
state
end
def initial_state
[]
end
def ev_is_subscript_change(event)
event =~ /MetadataUpdated/ && (ev_is_subscription(event) || ev_is_unsubscription(event))
end
def ev_is_subscription(event)
event =~ /Add subscriber./
end
def ev_is_unsubscription(event)
event =~ /Remove subscriber./
end
def ev_subscriber(event)
m = event.match(/net.tcp:\/\/([^:]*):/)
m && m[1]
end
def ev_timestamp(event)
m = event.match(/#{$TIMESTAMP}/)
m && DateTime.parse(m[0])
end
def add_subscriber(state, event)
(state + [ev_subscriber(event)]).uniq
end
def remove_subscriber(state, event)
state.reject{|e| e == ev_subscriber(event)}
end
def process_file(buffer, state)
events = log_to_events(buffer)
@index ||= 1;
events.select{|e| ev_is_subscript_change(e)}.inject(state){|st,ev| new_state = update(st, ev); puts "#{ev_timestamp(ev)}\t=DATEVALUE(MID(A#{@index},1,10))+TIMEVALUE(MID(A#{@index},12,8))\t#{new_state.length}"; @index += 1; @state = new_state; new_state}
end
def for_each_zip(path)
Dir.glob(path+"/*.Hosting.exe.*.zip") do |file|
Zip::File.open(file) do |zipfile|
zipfile.glob "*exe.log*" do |file|
yield zipfile.read(file)
end
end
end
end
@state = initial_state
for_each_zip('.../Server/Archive'){|f| process_file(f, @state) }
@NigelThorne
Copy link
Author

This was a quick hacky solution, but it worked. I'm posting it so I don't loose it in-case I one-day need it again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment