Skip to content

Instantly share code, notes, and snippets.

@ronan-mch
Last active August 29, 2015 14:01
Show Gist options
  • Save ronan-mch/e8e50d3068478739a921 to your computer and use it in GitHub Desktop.
Save ronan-mch/e8e50d3068478739a921 to your computer and use it in GitHub Desktop.
two shell scripts to parse a series of apache logs
# Start parse wrapper
num=$1
for i in $(seq 1 $num); do
echo "getting log for $i days ago";
./log_parser.sh "$i days ago";
done
# Start log parser
#!/bin/bash
#get timestamp for requested date
tstamp=$(date --date="$1" +"%Y%m%d")
#get name of log file and zip file for requested date log_file=access_log-$tstamp zip_file=$log_file.gz
#copy locally and unzip
echo "copying $log_file locally"
cp "/data/statistik/2014/logs/www-kb-dk/$zip_file" ./logs #gunzip "./logs/$zip_file"
echo "searching file for access pattern"
pattern_da="http://www.kb.dk/da/materialer/e-ressourcer/tidsskrifter/oldindex.html?subject=&q=&first=a&sso=true&reason=option"
pattern_en="http://www.kb.dk/en/materialer/e-ressourcer/tidsskrifter/oldindex.html?&reason=option"
zgrep "$pattern_da\|$pattern_en" "./logs/$zip_file" > temp
OPTION1=`grep --count option1 temp`
OPTION2=`grep --count option2 temp`
OPTION3=`grep --count option3 temp`
#output results
echo "================================"
echo "access for date: $tstamp"
echo "option1:$OPTION1"
echo "option2:$OPTION2"
echo "option3:$OPTION3"
echo "================================"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment