Skip to content

Instantly share code, notes, and snippets.

@andsel
Last active March 22, 2021 17:09
Show Gist options
  • Save andsel/dde3645cca583d2034554ba8aca65667 to your computer and use it in GitHub Desktop.
Save andsel/dde3645cca583d2034554ba8aca65667 to your computer and use it in GitHub Desktop.
Dump contents of Logstash checkpoint files
#!/usr/bin/env ruby
class CheckpointDecoder
attr_reader :path
def initialize(path)
@path = path
end
def decode
puts "filename, version, pageNum, firstUnackedPageNum, firstUnackedSeqNum, minSeqNum, eventCount, checksum"
if File.file?(path)
decode_single_file(path)
else
Dir.glob(path + '/checkpoint.*').sort.each { |file_name| decode_single_file(file_name) }
end
end
private
def decode_single_file(file_path)
file = File.open(file_path)
file_name = File.basename(file_path)
unpacked = file.read(34).unpack("S>NNLLNN")
version = unpacked[0]
pageNum = unpacked[1]
firstUnackedPageNum = unpacked[2]
firstUnackedSeqNum = unpacked[3]
minSeqNum = unpacked[4]
eventCount = unpacked[5]
checksum = unpacked[6]
puts "#{file_name}, #{version}, #{pageNum}, #{firstUnackedPageNum}, #{firstUnackedSeqNum}, #{minSeqNum}, #{eventCount}, #{checksum}"
end
end
if ARGV.length < 1
puts "Dump content of Logstash queue checkpoint files. CAn work on single file or on a folder full checkpoint files"
puts "examples: "
puts "single file: ./checkpoint_scanner.rb data/queue/main/checkpoint.585811 "
puts "folder: ./checkpoint_scanner.rb data/queue/main "
exit
end
checkpoint_decoder = CheckpointDecoder.new(ARGV[0])
checkpoint_decoder.decode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment