Skip to content

Instantly share code, notes, and snippets.

@jasonsee

jasonsee/gist:5366992

Created April 11, 2013 20:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jasonsee/5366992 to your computer and use it in GitHub Desktop.
Save jasonsee/5366992 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.apacheconf
#
# robots-dev.txt
#
# This is an alternate version of robots.txt that is used on dev and staging to prevent indexing of that content.
# .htaccess points users to this version on those hostnames.
#
User-agent: *
Crawl-delay: 10
# Directories
Disallow: /
# .htaccess file
..
<IfModule mod_rewrite.c>
..
# Redirect Google to the alternate robots.txt file that disallows indexing for dev/staging sites.
RewriteCond %{HTTP_HOST} ^dev\.paulbooker\.co.uk$ [NC]
RewriteRule ^robots.txt$ /robots-dev.txt [L,NC,R=301]
..
</IfModule>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment