Skip to content

Instantly share code, notes, and snippets.

@matsadler
Created November 9, 2012 12:51
Show Gist options
  • Save matsadler/4045530 to your computer and use it in GitHub Desktop.
Save matsadler/4045530 to your computer and use it in GitHub Desktop.
simple ruby sphinx query log parser
require "time"
# Usage:
#
# SphinxQueryLog.each("path/to/query.log") do |entry|
# # do stuff with entry
# end
#
class SphinxQueryLog
Entry = Struct.new(:date, :time, :mystery_attribute, :mode, :filters, :sort, :total, :offset, :limit, :group_by, :index, :ios, :kb, :ioms, :cpums, :query) do
def self.parse(line)
matchdata = line.match(%r{\A\[(?<date>[^\]]+)\] (?<time>[\d.]+) sec( (?<mystery_attribute>x2))? \[(?<mode>[a-z\d]+)/(?<filters>\d+)/(?<sort>[a-z+-]+) (?<total>\d+) \((?<offset>\d+),(?<limit>\d+)\)( (?<group_by>[a-z\d_@]+))?\] \[(?<index>[a-z\d_]+)\]( \[ios=(?<ios>[\d.]+) kb=(?<kb>[\d.]+) ioms=(?<ioms>[\d.]+) cpums=(?<cpums>[\d.]+)\])?( (?<query>.*))?\Z})
unless matchdata
warn "skipping #{line.inspect} as format unrecognised"
return nil
end
time = matchdata[:time].to_f
filters = matchdata[:filters].to_i
total = matchdata[:total].to_i
offset = matchdata[:offset].to_i
limit = matchdata[:limit].to_i
ios = matchdata[:ios].to_i
kb = matchdata[:kb].to_f
ioms = matchdata[:ioms].to_f
cpums = matchdata[:cpums].to_f
new(matchdata[:date], time, matchdata[:mystery_attribute], matchdata[:mode], filters, matchdata[:sort], total, offset, limit, matchdata[:group_by], matchdata[:index], ios, kb, ioms, cpums, matchdata[:query])
end
def date
Time === self[:date] ? self[:date] : self[:date] = Time.parse(self[:date])
end
end
def initialize(file)
if file.respond_to?(:to_s) && File.exist?(file.to_s)
@path = file.to_s
else
@file = file
end
end
def each
@file = File.open(@path) unless @file
if block_given?
@file.each_line do |line|
entry = Entry.parse(line.chomp)
yield entry if entry
end
self
else
Enumerator.new(self)
end
end
def self.each(file, &block)
new(file).each(&block)
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment