Skip to content

Instantly share code, notes, and snippets.

@iAugur
Last active January 8, 2019 13:33
Show Gist options
  • Save iAugur/6778020 to your computer and use it in GitHub Desktop.
Save iAugur/6778020 to your computer and use it in GitHub Desktop.
Excluding common requests from your apache logs

Original article: http://www.blue-bag.com/blog/excluding-common-requests-your-apache-logs

Log files can get filled up with repeated calls to files such as favicon, robots.txt, images, css js etc

Mostly you want to log the initial request for a page and not all of the resources subsequently requested.

Troubleshooting other issues may mean you would log files such as favicon, images etc - but generally they needlessly fill up your logs.

Do it ocassionally to look for missing images etc.

Here is a way to exclude them from your log files. This method sets an environment variable for each type of file so that you can turn logging on and off for each as you see fit.

E.g. To turn logging back on for images - update the line:

SetEnvIf Request_URI "(\.gif|\.png|\.jpg)$" image-request=nolog

to

SetEnvIf Request_URI "(\.gif|\.png|\.jpg)$" image-request=log

Put the following section in your Apache config / Vhosts settings for the site (just before the closing VirtualHost tag):

## flag robots.txt requests
SetEnvIf Request_URI "^/robots\.txt$" robots-request=log
## flag favicon requests
SetEnvIf Request_URI "^/favicon\.ico$" favicon-request=nolog
## flag image requests
SetEnvIf Request_URI "(\.gif|\.png|\.jpg)$" image-request=nolog
## flag Css and JS requests
SetEnvIf Request_URI \.css css-request=nolog
SetEnvIf Request_URI \.js js-request=nolog
## flag cron calls
SetEnvIf Request_URI "(^/cron\.php|^/bgp-start/)" cron-request=nolog
## set do_not_log if any of the above flags are set
SetEnvIf robots-request nolog do_not_log
SetEnvIf favicon-request nolog do_not_log
SetEnvIf image-request nolog do_not_log
SetEnvIf css-request nolog do_not_log
SetEnvIf js-request nolog do_not_log
SetEnvIf cron-request nolog do_not_log
## only log if do_not_log is not set
CustomLog /var/www/log/general-access.log vcommon env=!do_not_log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment