Skip to content

Instantly share code, notes, and snippets.

@MatthiasKunnen

MatthiasKunnen/rewrite-spiders.htaccess

Created November 9, 2017 19:24
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save MatthiasKunnen/4a962076a4bcf73122e6f913451f9ce2 to your computer and use it in GitHub Desktop.
Save MatthiasKunnen/4a962076a4bcf73122e6f913451f9ce2 to your computer and use it in GitHub Desktop.
Download ZIP
Rewrite spider to static HTML files
Raw
rewrite-spiders.htaccess
RewriteEngine On
# Remove trailing /
RewriteRule ^(.*)/$ /$1 [L,R=301]
# Rewrite spiders to static html
RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|msnbot|yahoo|Baidu|aolbuild|facebookexternalhit|iaskspider|DuckDuckBot|Applebot|Almaden|iarchive|archive.org_bot) [NC]
RewriteCond %{DOCUMENT_ROOT}/static%{REQUEST_URI}.html -f
RewriteRule ^(.*)$ /static/$1.html [L]
# Rewrite spiders to static index.html
RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|msnbot|yahoo|Baidu|aolbuild|facebookexternalhit|iaskspider|DuckDuckBot|Applebot|Almaden|iarchive|archive.org_bot) [NC]
RewriteCond %{REQUEST_URI} "^/$"
RewriteRule ^ /static/index.html [L]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment