Skip to content

Instantly share code, notes, and snippets.

@mspivak mspivak/.htaccess
Created Nov 29, 2012

What would you like to do?
Different robots.txt files for HTTP and HTTPS
RewriteEngine On
#This will have the file processed by robots.php instead of a regular static file.
RewriteRule ^robots.txt$ /robots.php [L]
header("Content-Type: text/plain; charset=utf-8");
$protocol = $_SERVER['HTTPS'] ? 'HTTPS' : 'HTTP';
echo file_get_contents(dirname(__FILE__).'/robots.txt.'.strtolower($protocol));
User-agent: *
User-agent: *
Disallow: /

This comment has been minimized.

Copy link
Owner Author

mspivak commented Nov 29, 2012

This tells robot crawlers to read every HTTP path and none HTTPS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.