Skip to content

Instantly share code, notes, and snippets.

@thoop
Last active January 27, 2024 14:07
Show Gist options
  • Save thoop/8072354 to your computer and use it in GitHub Desktop.
Save thoop/8072354 to your computer and use it in GitHub Desktop.
Official prerender.io .htaccess for Apache.
# Change YOUR_TOKEN to your prerender token
# Change http://example.com (at the end of the last RewriteRule) to your website url
<IfModule mod_headers.c>
RequestHeader set X-Prerender-Token "YOUR_TOKEN"
</IfModule>
<IfModule mod_rewrite.c>
RewriteEngine On
<IfModule mod_proxy_http.c>
RewriteCond %{HTTP_USER_AGENT} googlebot|bingbot|yandex|baiduspider|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator [NC,OR]
RewriteCond %{QUERY_STRING} _escaped_fragment_
# Only proxy the request to Prerender if it's a request for HTML
RewriteRule ^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff|\.svg))(.*) http://service.prerender.io/http://example.com/$2 [P,L]
</IfModule>
</IfModule>
@yokoishioka
Copy link

you are awesome!! i almost wept when i finally saw the token not used message go away. thanks!!

@arnotixe
Copy link

Just adding 2 cents:
On my hosting provider (Dreamhost), the [NC,OR] yielded strange results when just copy/pasting. On RewriteCond with only one line, I had to write it as
RewriteCond … googlebot|yandex|…|…|… [NC]
(re-writing the RewriteCond with one per line also worked, but not with [OR] on the first line. This would work:)

RewriteCond … googlebot [NC]
RewriteCond … yandex [NC,OR]
RewriteCond … WhatsApp [NC,OR]

note the seemingly missing OR on the first line

On the other hand, my second cent is that last WhatsApp entry - turns out WhatsApp does the scraping directly from inside the app (at least it did on my android phone today ;) So my full line is now
RewriteCond %{HTTP_USER_AGENT} googlebot|bingbot|yandex|baiduspider|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|WhatsApp [NC]

(And my full htaccess

    RewriteEngine On
        
        # https://stackoverflow.com/questions/18406156/redirect-all-to-index-php-using-htaccess
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteCond %{HTTP_USER_AGENT} googlebot|bingbot|yandex|baiduspider|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|WhatsApp [NC]
        #        RewriteCond %{HTTP_USER_AGENT} facebookexternalhit|googlebot [NC]   MUST BE WRITTEN WITHOUT OR
        #        RewriteCond %{HTTP_USER_AGENT} googlebot [NC]
        #        RewriteCond %{HTTP_USER_AGENT} facebookexternalhit [NC,OR]          'OR' IS FOR SECOND LINE (AND THIRD AND FOURTH ETC. WON'T WORK ON FIRST LINE)
        RewriteRule ^(.*)$ opengraph.php?q=$1 [NC,L,QSA]

        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteRule ^(.*)$ redir.php?orig_path=$1 [NC,L,QSA]

</IfModule>

@tyluudinh
Copy link

Angular 7 + Apache2: work for me. Ping me if you have any issue

.htaccess: https://gist.github.com/tyluudinh/2149d6cc62219c141f619682b5378c47

.apache2.conf: https://gist.github.com/tyluudinh/bc257a1aa2a4b8826ff28916238cc27d

@Kotstar1
Copy link

@tyluudinh This worked for me, thank you 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment