Skip to content

Instantly share code, notes, and snippets.

@everfluxx
everfluxx / .htaccess
Created November 2, 2011 16:44
mod_rewrite rule set to block major search engine crawlers sending POST requests
### Prevent major search engine crawlers from sending POST requests ###
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_USER_AGENT} Slurp [OR]
RewriteCond %{HTTP_USER_AGENT} bingbot
RewriteCond %{REQUEST_METHOD} =POST
RewriteRule .* - [env=set_allow_header:true,R=405,L]
Header set Allow "GET, HEAD" env=set_allow_header
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{QUERY_STRING} .
RewriteRule .* - [env=send_x_robot:true]
Header set X-Robots-Tag "noindex" env=send_x_robot
</IfModule>