Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@hieblmedia
Last active June 8, 2021 09:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hieblmedia/c09fce90f03b5ed8cd2ed51bf78774d2 to your computer and use it in GitHub Desktop.
Save hieblmedia/c09fce90f03b5ed8cd2ed51bf78774d2 to your computer and use it in GitHub Desktop.
no-index header (X-Robots-Tag) + robots.txt for Develop/Testing/Stating subdomains
# Set X-Robots-Tag
<If "%{HTTP_HOST} =~ /^(develop|dev|test|testing|stage|staging)\./">
Header set X-Robots-Tag "noindex, nofollow"
</If>
<IfModule mod_rewrite.c>
# Enable URL rewriting
RewriteEngine On
# ....
# Response robots_no.txt on robots.txt request for disallow contents (don't forget to put robots_no.txt in directory root)
RewriteCond %{HTTP_HOST} ^(develop|dev|test|testing|stage|staging)\.
RewriteRule robots\.txt$ robots_no.txt [NS]
# ....
</IfModule>
User-Agent: *
Disallow: /
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment