Skip to content

Instantly share code, notes, and snippets.

@lidio601

lidio601/.htaccess

Created May 6, 2015 10:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lidio601/036f080b21e040533616 to your computer and use it in GitHub Desktop.
Save lidio601/036f080b21e040533616 to your computer and use it in GitHub Desktop.
Download ZIP
.htaccess file to disallow access to clients that aren't search engine spiders
Raw
.htaccess
# http://www.robotstxt.org/robotstxt.html
# http://httpd.apache.org/docs/2.0/misc/rewriteguide.html
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !bot
RewriteRule ^robots.txt - [L,R=404]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment