Skip to content

Instantly share code, notes, and snippets.

@camathieu
Last active August 29, 2015 14:06
Show Gist options
  • Save camathieu/f9b90ea05f933d8c4d65 to your computer and use it in GitHub Desktop.
Save camathieu/f9b90ea05f933d8c4d65 to your computer and use it in GitHub Desktop.
Parse hbase regions server logs
#!/bin/bash
###
# Charles-Antoine Mathieu <skatkatt@root.gg>
#
# Parse memstore flushes in hbase rs logs (tested against hbase 0.96)
# $1 : rs log file
# $2 : command : flush | major
###
if [ "$2" == "flush" ];then
cat $1 | grep "memstore flush" | sed -n 's/^\(.*\) INFO.*of \(.*\)\/[0-9]\+, current.*region \(.*\),\(.*\),14.*in \(.*\), seq.*$/\1 \3 \4 \2 \5/p'
fi
if [ "$2" == "minor" ];then
cat $1 | grep "Completed compaction" | sed -n 's/^\(.*\) INFO.*regionName=\(.*\),\(.*\),.*, storeName=.*fileCount=\([0-9]\+\),.*fileSize=\(.*\), priority.*duration=\(.*\).*$/\1 \2 \4 files for \5 in \6/p'
fi
if [ "$2" == "major" ];then
cat $1 | grep "Completed major compaction" | sed -n 's/^\(.*\) INFO.*\([0-9]\+\) file(s) in data of \(.*\),\(.*\),.*into.*total size for store is \(.*\). This selection was in queue for \(.*\), and took \(.*\) to execute.$/\1 \3 \2 files for \5 in \7 ( queued for \6 )/p'
fi
if [ "$2" == "split" ];then
cat $1 | grep 'Region split' | sed -n 's/^\(.*\) INFO.*Parent=\(.*\),\(.*\),.*, new regions.*Split took \(.*\)$/\1 \2 in \4/p'
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment