Skip to content

Instantly share code, notes, and snippets.

@chadclark

chadclark/Robots Environment .htaccess

Last active October 17, 2022 11:48
Show Gist options
  • Star 10 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save chadclark/5600194 to your computer and use it in GitHub Desktop.
Save chadclark/5600194 to your computer and use it in GitHub Desktop.
Download ZIP
Robots.txt for Staging and Production. By adding these rewrite rules to your .htaccess file, robots_dev.txt will be served as robots.txt on any non-production server.
Raw
Robots Environment .htaccess
RewriteEngine On
# Robots.txt for Staging and Production -- change productiondomain.com to the actual url of your production site
RewriteCond %{HTTP_HOST} !productiondomain.com$ [NC]
RewriteRule ^robots.txt robots_dev.txt [L]
Raw
robots.txt
User-agent: *
Allow: /
Raw
robots_dev.txt
User-agent: *
Disallow: /
@dalph
Copy link

dalph commented Jan 23, 2020

Maybe, it is more careful solution:

RewriteCond %{HTTP_HOST} !^productiondomain.com $ [NC]
RewriteRule ^robots.txt robots_dev.txt [L]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment