Last active

Embed URL

HTTPS clone URL

SSH clone URL

You can clone with HTTPS or SSH.

Download Gist

Example Apache configuration for serving pre-rendered HTML from Javascript pages/apps using the Prerender Service at https://github.com/collectiveip/prerender

View .htaccess
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /prerender-test/
 
# Virtual/pushState URIs (requests not matching an existing file get forwarded to index.html)
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* index.html [L]
 
<IfModule mod_proxy_http.c>
# Enable prerendering for .html and directory index files
RewriteCond %{HTTP_USER_AGENT} googlebot|yahoo|bingbot|baiduspider [NC,OR]
RewriteCond %{QUERY_STRING} _escaped_fragment_|prerender=1
RewriteCond %{HTTP_USER_AGENT} !^Prerender
RewriteRule ^(.*\.html)?$ http://prerender.herokuapp.com/http://www.yourdomain.com/prerender-test/$1 [P,L]
</IfModule>
</IfModule>
 
# Optionally add a ProxyPassReverse directive to ensure that 301/302 redirects
# issued by the prerender service are correctly forwarded to the client.
# Note that this must be located in your Apache config rather than .htaccess
 
# <IfModule mod_proxy_http.c>
# ProxyPassReverse /prerender-test/ http://prerender.herokuapp.com/http://www.yourdomain.com/prerender-test/
# </IfModule>

Here is a more extensive list of some of the latest bot user agents sourced from http://user-agent-string.info/list-of-ua/bots

Googlebot|bingbot|Googlebot-Mobile|Baiduspider|Yahoo|YahooSeeker|DoCoMo|Twitterbot|TweetmemeBot|Twikle|Netseer|Daumoa|SeznamBot|Ezooms|MSNBot|Exabot|MJ12bot|sogou\sspider|YandexBot|bitlybot|ia_archiver|proximic|spbot|ChangeDetection|NaverBot|MetaJobBot|magpie-crawler|Genieo\sWeb\sfilter|Qualidator.com\sBot|Woko|Vagabondo|360Spider|ExB\sLanguage\sCrawler|AddThis.com|aiHitBot|Spinn3r|BingPreview|GrapeshotCrawler|CareerBot|ZumBot|ShopWiki|bixocrawler|uMBot|sistrix|linkdexbot|AhrefsBot|archive.org_bot|SeoCheckBot|TurnitinBot|VoilaBot|SearchmetricsBot|Butterfly|Yahoo!|Plukkie|yacybot|trendictionbot|UASlinkChecker|Blekkobot|Wotbox|YioopBot|meanpathbot|TinEye|LuminateBot|FyberSpider|Infohelfer|linkdex.com|Curious\sGeorge|Fetch-Guess|ichiro|MojeekBot|SBSearch|WebThumbnail|socialbm_bot|SemrushBot|Vedma|alexa\ssite\saudit|SEOkicks-Robot|Browsershots|BLEXBot|woriobot|AMZNKAssocBot|Speedy|oBot|HostTracker|OpenWebSpider|WBSearchBot|FacebookExternalHit

Owner

Thanks for the list @homerjam!

Hey guys, this isn't working for me. The request hits prerender but everything gets loaded into prerender as index.html and returns a blank html page: <html><head></head><body></body></html> (I am using User-Agent Switcher as a chrome extension to test behavior).

Before I switched to the pushState method, prerender was working with the "#!" method and I didn't need the following:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* index.html [L]

Thoughts?

Hey again. Just wanted to let you guys know that I got things figured out but the fallback URL condition I listed above needed to be below the prerender proxy rewrite rule or else the page falls back to index.html before there is a chance to send the url to prerender.

Owner

FYI:
Googlebot is starting to render Javascript so you may want to remove it from the user agents list (after testing in webmaster tools)

See:
http://googlewebmastercentral.blogspot.com/2014/05/rendering-pages-with-fetch-as-google.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.