Skip to content

Instantly share code, notes, and snippets.

@mspivak mspivak/.htaccess
Created Nov 29, 2012

Embed
What would you like to do?
Different robots.txt files for HTTP and HTTPS
RewriteEngine On
#This will have the file processed by robots.php instead of a regular static file.
RewriteRule ^robots.txt$ /robots.php [L]
<?php
header("Content-Type: text/plain; charset=utf-8");
$protocol = $_SERVER['HTTPS'] ? 'HTTPS' : 'HTTP';
echo file_get_contents(dirname(__FILE__).'/robots.txt.'.strtolower($protocol));
User-agent: *
Disallow:
User-agent: *
Disallow: /
@mspivak

This comment has been minimized.

Copy link
Owner Author

commented Nov 29, 2012

This tells robot crawlers to read every HTTP path and none HTTPS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.