Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
#!/bin/bash
# variables
LOGFILE="/var/log/nginx/access.log"
LOGFILE_GZ="/var/log/nginx/access.log.*"
RESPONSE_CODE="200"
# functions
filters(){
grep $RESPONSE_CODE \
| grep -v "\/rss\/" \
| grep -v robots.txt \
| grep -v "\.css" \
| grep -v "\.jss*" \
| grep -v "\.png" \
| grep -v "\.ico"
}
filters_404(){
grep "404"
}
request_ips(){
awk '{print $1}'
}
request_method(){
awk '{print $6}' \
| cut -d'"' -f2
}
request_pages(){
awk '{print $7}'
}
wordcount(){
sort \
| uniq -c
}
sort_desc(){
sort -rn
}
return_kv(){
awk '{print $1, $2}'
}
request_pages(){
awk '{print $7}'
}
return_top_ten(){
head -10
}
## actions
get_request_ips(){
echo ""
echo "Top 10 Request IP's:"
echo "===================="
cat $LOGFILE \
| filters \
| request_ips \
| wordcount \
| sort_desc \
| return_kv \
| return_top_ten
echo ""
}
get_request_methods(){
echo "Top Request Methods:"
echo "===================="
cat $LOGFILE \
| filters \
| request_method \
| wordcount \
| return_kv
echo ""
}
get_request_pages_404(){
echo "Top 10: 404 Page Responses:"
echo "==========================="
zgrep '-' $LOGFILE $LOGFILE_GZ\
| filters_404 \
| request_pages \
| wordcount \
| sort_desc \
| return_kv \
| return_top_ten
echo ""
}
get_request_pages(){
echo "Top 10 Request Pages:"
echo "====================="
cat $LOGFILE \
| filters \
| request_pages \
| wordcount \
| sort_desc \
| return_kv \
| return_top_ten
echo ""
}
get_request_pages_all(){
echo "Top 10 Request Pages from All Logs:"
echo "==================================="
zgrep '-' --no-filename $LOGFILE $LOGFILE_GZ \
| filters \
| request_pages \
| wordcount \
| sort_desc \
| return_kv \
| return_top_ten
echo ""
}
# executing
get_request_ips
get_request_methods
get_request_pages
get_request_pages_all
get_request_pages_404
@hsafe

This comment has been minimized.

Copy link

commented Jan 14, 2018

Made my life easy...Thanks
one thing though...need some help with zgrep '-' --no-filename

@gbot

This comment has been minimized.

Copy link

commented Mar 9, 2018

Hi there, nice script, thanks for gist-ing it!

I'm trying to get this to work with a ServerPilot.io server, scanning all logs in all users app directories.

I've changed the $LOGFILE and $LOGFILE_GZ line to this:

LOGFILE="/srv/users/*/log/*/*nginx.access.log*"
LOGFILE_GZ="/srv/users/*/log/*/*nginx.access.log*.gz"

... but I'm not sure that it's doing what I need it to do... (sorry, hopeless!) Do I need to loop over the $LOGFILES to get this to work?

Thanks again!

@panqiincs

This comment has been minimized.

Copy link

commented Jun 18, 2018

Hello, I think grep $RESPONSE_CODE should be changed to awk '$9==200', because the IP address may contain the number 200.

@marcbrunet

This comment has been minimized.

Copy link

commented May 19, 2019

fix litet error in scan 404 and 200 resposnes view in my fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.