# Change YOUR_TOKEN to your prerender token and uncomment that line if you want to cache urls and view crawl stats | |
# Change example.com (server_name) to your website url | |
# Change /path/to/your/root to the correct value | |
server { | |
listen 80; | |
server_name example.com; | |
root /path/to/your/root; | |
index index.html; | |
location / { | |
try_files $uri @prerender; | |
} | |
location @prerender { | |
#proxy_set_header X-Prerender-Token YOUR_TOKEN; | |
set $prerender 0; | |
if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") { | |
set $prerender 1; | |
} | |
if ($args ~ "_escaped_fragment_") { | |
set $prerender 1; | |
} | |
if ($http_user_agent ~ "Prerender") { | |
set $prerender 0; | |
} | |
if ($uri ~* "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") { | |
set $prerender 0; | |
} | |
#resolve using Google's DNS server to force DNS resolution and prevent caching of IPs | |
resolver 8.8.8.8; | |
if ($prerender = 1) { | |
#setting prerender as a variable forces DNS resolution since nginx caches IPs and doesnt play well with load balancing | |
set $prerender "service.prerender.io"; | |
rewrite .* /$scheme://$host$request_uri? break; | |
proxy_pass http://$prerender; | |
} | |
if ($prerender = 0) { | |
rewrite .* /index.html break; | |
} | |
} | |
} |
This comment has been minimized.
This comment has been minimized.
You're right, thanks. I'll update that. |
This comment has been minimized.
This comment has been minimized.
AttilaSATAN
commented
May 4, 2014
How about html5mode of angularjs? Is prerender supports html5mode, would it be enough to add googlebot to the if statement?
|
This comment has been minimized.
This comment has been minimized.
Google's recommended way is to use escaped_fragment. In theory, yes we should add it to the user agent check, but in reality we wouldn't want Google to request the same URL using another user agent and then think the user is cloaking because the content is different. So we try to stay on the safe side. |
This comment has been minimized.
This comment has been minimized.
sansmischevia
commented
Jul 2, 2014
Confused... why isn't googlebot listed? |
This comment has been minimized.
This comment has been minimized.
Michal-sk
commented
Jul 14, 2014
@sansmischevia : because googlebot uses the escaped_fragment, which is listed. |
This comment has been minimized.
This comment has been minimized.
jamiel
commented
Aug 26, 2014
If users are using HTML5 pushState, surely Google will request the URL's without escaped_fragment ? |
This comment has been minimized.
This comment has been minimized.
toamitkumar
commented
Sep 4, 2014
Search bots look for --> |
This comment has been minimized.
This comment has been minimized.
sentient
commented
Sep 18, 2014
I'm not too sure on
Am I only replacing the example.com ? Could we not use the $server_name variable for this?
|
This comment has been minimized.
This comment has been minimized.
akoumjian
commented
Sep 24, 2014
If our "location /" block is where we typically specify a reverse proxy with proxy_pass, should I assume we would essentially add that to the "if (@prerender = 0)" section? |
This comment has been minimized.
This comment has been minimized.
@sentient good idea :) @akoumjian yes, in that case you would do your own proxy_pass "if @prerender = 0" |
This comment has been minimized.
This comment has been minimized.
I added this since we were seeing issues where nginx was caching IPs and hitting servers that might have been taken out of our load balancer rotation:
|
This comment has been minimized.
This comment has been minimized.
patrickng
commented
Oct 1, 2014
With nginx 1.4.6 (ubuntu) I get an error when I try to restart nginx service or reload the configuration.
I had to move resolver out of the conditional for it to work but this isn't ideal. Any ideas? |
This comment has been minimized.
This comment has been minimized.
fergusg
commented
Oct 4, 2014
I'm wondering if
There can be multiple |
This comment has been minimized.
This comment has been minimized.
cjroebuck
commented
Oct 15, 2014
@thoop @patrickng I too have the same issue ('resolver' directive is not allowed here) |
This comment has been minimized.
This comment has been minimized.
Sorry, I wish github would send notifications for comments on gists :( @fergusg we used to have it set to $host but recently changed it to $server_name. I'll look into a better solution for the resolver in the if-conditional. |
This comment has been minimized.
This comment has been minimized.
evityuk
commented
Oct 21, 2014
Vkontakte social network uses this - "Mozilla/5.0 (compatible; vkShare; +http://vk.com/dev/Share)"
See embeding docs. Sharing uses the same userAgent |
This comment has been minimized.
This comment has been minimized.
evandhoffman
commented
Oct 23, 2014
If behind a load balancer, the $scheme var may not be set right - if the LB is doing SSL termination, the scheme on the machine behind the box may be http. Prerender service would then try to access http://, but get bounced to https://, which in my case did not work. I had to hard-code https:// in there. |
This comment has been minimized.
This comment has been minimized.
leorue
commented
Nov 3, 2014
Any update on @patrickng resolver issue? I took out the conditional as well. |
This comment has been minimized.
This comment has been minimized.
@leorue can you try the new nginx config? I just updated it to move the resolver outside of the "if" statement. |
This comment has been minimized.
This comment has been minimized.
I just changed $server_name back to $host. Hopefully that clears up any issues with the server name not being the actual url of your site. |
This comment has been minimized.
This comment has been minimized.
shirokoweb
commented
Nov 30, 2014
Hi, do I have to install something on my webserver running nginx as frontend HTTP server, or do I simply need to add this snippet to my vhost .conf ? |
This comment has been minimized.
This comment has been minimized.
You should just be able to add this snippet to your .conf file. Email me at todd@prerender.io if you're having any problems with it. Github doesn't send notifications on gists so I'll be able to help you more quickly over email. |
This comment has been minimized.
This comment has been minimized.
varuzhnikov
commented
Dec 5, 2014
http://wiki.nginx.org/IfIsEvil Directive if has problems when used in location context, in some cases it doesn't do what you expect but something completely different instead. In some cases it even segfaults. It's generally a good idea to avoid it if possible. The only 100% safe things which may be done inside if in location context are: return ...; It is important to note that the behaviour of if is not inconsistent, given two identical requests it will not randomly fail on one and work on the other, with proper testing and understanding ifs can be used. The advice to use other directives where available still very much apply, though. |
This comment has been minimized.
This comment has been minimized.
intellix
commented
Dec 8, 2014
Google doesn't use ?escaped_fragment= for all of it's services. It might do for it's indexer but for instance when I use it from the Webmaster tools "Fetch as Google", it correctly renders it but the HTML it received was before rendering:
In fact, I am ALWAYS seeing two requests from Googlebot - One with and without the escaped fragment. A few that I added myself: |
This comment has been minimized.
This comment has been minimized.
csbenjamin
commented
Dec 8, 2014
@intellix I guess that in the first request google find the and then do a second request with escaped_fragment. Before the first request it did not know that it needs to use escaped_fragment |
This comment has been minimized.
This comment has been minimized.
@varuzhnikov if we could refactor to remove if statements, that would be ideal. Any idea on the best way to do that? We haven't had any problems with this configuration yet. @intellix @csbenjamin is correct. The first request see's your tag and re-requests the page with the escaped fragment parameter. "Fetch as Google" does not follow the escaped fragment, and that is a known bug. If you are seeing services that do not send two requests (one with escaped fragment), then we should add them to the whitelist of user agents. |
This comment has been minimized.
This comment has been minimized.
bastijan
commented
Dec 14, 2014
I use self hosted prerender and nginx server/angularjs html application but when try share contentn on facebook or linked I always got root page (index.html) / nginx conf just for review: in app.js $locationProvider.hashPrefix('!'); and in index.html template |
This comment has been minimized.
This comment has been minimized.
lethaldose
commented
Dec 22, 2014
we had a AWS nginx proxy setup and were having timeout issues - service.prerender.io could not be resolved (60: Operation timed out). We were able to get around this by using nginx upstream:
and the if in location block changed to:
|
This comment has been minimized.
This comment has been minimized.
creatorkuang
commented
Dec 26, 2014
I use the official server :"service.prerender.io" and it works, but when i try to use my own server ,it didn't. I have test it with http://myserver.com/http://www.google.com and it work with my server which i think it mean the server work. But when i replace "service.prerender.io" with "myserver.com" , it didn't work and got an error with 502 Bad Gateway . Anyone know why? |
This comment has been minimized.
This comment has been minimized.
geolan80
commented
Jan 17, 2015
I use nginx like a reverse proxy and i would like to use prerender. I don't know how to adapt the nginx.conf especially this section: location / { I can't use the "try_files" instruction as i use "proxy_pass" instruction. Do you know how to do ? Thank you |
This comment has been minimized.
This comment has been minimized.
rkulla
commented
Jan 17, 2015
@creatorkuang it'd be hard to know without seeing your configuration. myserver.com needs to be running the open-source prerender node server and on the right port (by default it's 3000), and your nginx prerender middleware has to be configured to proxy there if the prerender variable = 1. E.g. |
This comment has been minimized.
This comment has been minimized.
rkulla
commented
Jan 17, 2015
@geolart I think some people above commented that you can do that by moving your proxy_pass line into the |
This comment has been minimized.
This comment has been minimized.
geolan80
commented
Jan 18, 2015
@rkulla Thanks to you it's work for me ! |
This comment has been minimized.
This comment has been minimized.
rkulla
commented
Jan 18, 2015
When also using nginx as a reverse proxy via |
This comment has been minimized.
This comment has been minimized.
Send me an email at support@prerender.io if anyone has any issues. I don't get notified when someone comments on this gist. |
This comment has been minimized.
This comment has been minimized.
BenjaminPrice
commented
Apr 2, 2015
As per Facebook official documentation, you should also add the user agent 'facebot' to $http_user_agent https://developers.facebook.com/docs/sharing/best-practices#crawl |
This comment has been minimized.
This comment has been minimized.
colthreepv
commented
Apr 8, 2015
I have monitored many requests that come with There is an exhaustive of those User-Agents somewhere? |
This comment has been minimized.
This comment has been minimized.
jstoiko
commented
Apr 28, 2015
You might want to add 'svg' to the list of extensions not prerendered. |
This comment has been minimized.
This comment has been minimized.
ermakovich
commented
May 15, 2015
@evandhoffman we have the same issue when hosting on Heroku, which in turn seems to be using Amazon ELB. |
This comment has been minimized.
This comment has been minimized.
@MrGamer you don't want to add Googlebot (or any other crawlers that support the escaped fragment protocol) to the user agent list. You could get penalized for cloaking. You want Google to continue using the escaped fragment protocol @ermakovich great idea! Send me an email at support@prerender.io if anyone has any issues. I don't get notified when someone comments on this gist. |
This comment has been minimized.
This comment has been minimized.
ashishgupta2
commented
Jun 25, 2015
For nginx proxy_pass users: do two things.
replace #rewrite .* /index.html break; with your proxy_pass as shown below (my application is running on port 3009). proxy_pass http://127.0.0.1:3009; |
This comment has been minimized.
This comment has been minimized.
matfin
commented
Aug 31, 2015
Does anyone have an example of something like this working for Meteor ? |
This comment has been minimized.
This comment has been minimized.
blaind
commented
Sep 13, 2015
@matfin see https://gist.github.com/blaind/12e53d5d9aa77c9fd841 for a meteor config. Had to use global proxy configs for both (can't put them in if's), but prerender.io seems to accept upgraded HTTP 1.1 too. |
This comment has been minimized.
This comment has been minimized.
maxklenk
commented
Sep 15, 2015
Please add |
This comment has been minimized.
This comment has been minimized.
alphastorm
commented
Oct 1, 2015
There's also trailing whitespace in 6 lines, might want to remove that. |
This comment has been minimized.
This comment has been minimized.
donny08
commented
Oct 16, 2015
@thoop I'm configuring prerender using nginx with nodejs backened server.Below is my nginx.conf file:- server {
But I am seeing JavaScript and not the static HTML.I had the checked Common Problem section,It has mentioned Try moving your Prerender middleware configuration higher in your file,I want to know what is |
This comment has been minimized.
This comment has been minimized.
thabofletcher
commented
Oct 16, 2015
I've posted a small project that demonstrates an implementation of how to use this without violating the "If Is Evil" principle: This allows any configuration for your app, including rewrites, try_files, and proxy_path, all outside if blocks. I believe it can be consolidated into a single gist/config, but I prefer the use of includes for readability. |
This comment has been minimized.
This comment has been minimized.
donny08
commented
Oct 27, 2015
This is my url :http://example.com/customer/#/index,I have already done prerender configuration.How do i access the prerender page of http://example.com/customer/#/index, what should be my url format |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
asquel
commented
Nov 27, 2015
+1 for the question from @thiagofesta ! (what do you think @thoop ?) |
This comment has been minimized.
This comment has been minimized.
TheProdigyFilippo
commented
Nov 29, 2015
@thiagofesta @asquel don't believe in this. It's not as nice as google tells. We shot ourselves in feets when we changed our static title and meta description into dynamic using AngularJS Correction: So when you depend on google rendering only, you should remember to delete |
This comment has been minimized.
This comment has been minimized.
antoineco
commented
Nov 29, 2015
@thabofletcher I used a similar configuration but when I got rid of
and replaced it by a single edit: the explanation is here, IfIsEvil (nginx docs) |
This comment has been minimized.
This comment has been minimized.
rogierslag
commented
Jan 21, 2016
I've made a fork and updated it with the following user agents These now also include stuff as Viber, WhatsApp, and Telegram which all try to parse the og tags |
This comment has been minimized.
This comment has been minimized.
charlesdg
commented
Jan 22, 2016
@ermakovich thanks for the hack for heroku! If you are using the Heroku Buildpack: NGINX (https://elements.heroku.com/buildpacks/ryandotsmith/nginx-buildpack), just replace rewrite .* /$scheme://$host$request_uri? break; by rewrite .* /$http_x_forwarded_proto://$host$request_uri? break; Hope that will be useful ! |
This comment has been minimized.
This comment has been minimized.
kascote
commented
Jan 26, 2016
I'm using this setup with an Angular application and works ok except to for the root (/) path that never goes throw prerender. someone have this issue ? how to resolve it ? |
This comment has been minimized.
This comment has been minimized.
intellix
commented
Jan 31, 2016
Been having issues with prerender dying and having my entire site disappear from Google when everything returns 504 errors (my fault for not having server alerts of any kind). I think in such a case where prerender just dies, we should have a fallback so google at least gets served your HTML/JS (they just won't for API calls to resolve, as prerender is there for). Example error that'll bring your prerender service down
Been playing around with how to get error_page working with a proxy and this is what works.: server {
# Add this line, to redirect 50x errors back to index.html
error_page 500 502 503 504 =200 /index.html;
location / {
index index.html;
try_files $uri @prerender;
}
location @prerender {
# Add this line inside @prerender location
proxy_intercept_errors on;
}
} |
This comment has been minimized.
This comment has been minimized.
thedug
commented
Feb 4, 2016
Why not just simply do this vs. the rewrite? proxy_pass http://service.prerender.io/$scheme://$host$request_uri?; |
This comment has been minimized.
This comment has been minimized.
marcelpanse
commented
Feb 22, 2016
Google has deprecated the escaped fragment scheme: I think googlebot should be added to the whitelist, the escaped fragment scheme will probably still used by other bots like bing. |
This comment has been minimized.
This comment has been minimized.
@marcelpanse Google is still not doing a good enough job when crawling javascript, which they even admitted in their last Google Webmaster Hangout. Google still follows the escaped fragment crawling protocol like they did before so Prerendering pages for Google is still the recommended way of doing things. Adding googlebot to the whitelist could get your website penalized for cloaking so it's not recommended. You should support the escaped fragment crawling protocol for Googlebot. |
This comment has been minimized.
This comment has been minimized.
@thedug The rewrite is the way that it is because nginx caches IP addresses. If an IP address changes on our end, then your prerendering could stop working. The way we have it with the variable will prevent nginx from caching IP addresses to prevent this problem. |
This comment has been minimized.
This comment has been minimized.
I don't get any notifications when people comment on this thread so please email me at todd@prerender.io if anyone has any questions. |
This comment has been minimized.
This comment has been minimized.
ray65536
commented
Mar 11, 2016
You should move block of Like this:
|
This comment has been minimized.
This comment has been minimized.
SiddharthTerse
commented
Mar 26, 2016
Hi, If i am browsing any webpage so 1st time with ?_escaped_fragment redirecting 301 & from next time 200 OK . this is happening for all pages.... My nginx config file for prerender is... location @prerender {
} |
This comment has been minimized.
This comment has been minimized.
SiddharthTerse
commented
Mar 28, 2016
@thoop, i found article where google disclosed they have stopped to support official for escaped_fragment (http://searchengineland.com/google-has-deprecated-their-ajax-crawling-scheme-233402) . So now what is the alternative for google crowling now ? |
This comment has been minimized.
This comment has been minimized.
wellington1993
commented
Mar 30, 2016
Thanks in a lot! |
This comment has been minimized.
This comment has been minimized.
bwoodlt
commented
Mar 30, 2016
Hi Guys, I'm trying to use this config, but keep returning 404 error. I'll appreciate any help in having a quick look. server {
} |
This comment has been minimized.
This comment has been minimized.
rickyk586
commented
Apr 6, 2016
lethaldose's solution above worked for me |
This comment has been minimized.
This comment has been minimized.
Gbeschbacher
commented
Apr 12, 2016
Hey there, i am trying to use prerender with nginx and our angular.js app. This is the nginx-config which proxies to the docker-container
and this is the currently working nginx-config inside the container:
i tried to get it to work like this but it doesnt:
Thanks in advance! |
This comment has been minimized.
This comment has been minimized.
vic3lord
commented
Apr 25, 2016
if other people like me tried to proxy_pass if $prerender = 0 then just remove the if condition as it acts like else Hope it helps |
This comment has been minimized.
This comment has been minimized.
AntonAL
commented
Apr 26, 2016
•
Hello, guys! I've started prerender server listening port 3000, in my production machine, used this config file.
When I requesting home page from external client host, I'm always getting 504 and no rendering:
When I requesting home page directly from production server, everything is working: When I using It seems, like I have problem with local prerenderer… Also, I don't understand, how this configuration passes requests to meteor in case of non-bot.
It loads static index.html…but we are using Meteor… My initial configuration of nginx without prerender is taken from guide How To Deploy a Meteor.js Application on Ubuntu 14.04 with Nginx Anyone having the same issue? |
This comment has been minimized.
This comment has been minimized.
aaronbuchanan
commented
May 12, 2016
@intellix can you shed some light on prerender dying on your site? Know what the root cause was? Find a resolution you're happy with? |
This comment has been minimized.
This comment has been minimized.
tpoxa
commented
Jun 3, 2016
What about php-fpm? |
This comment has been minimized.
This comment has been minimized.
partounian
commented
Jul 2, 2016
Yes, how could we make this play nicely with PHP? |
This comment has been minimized.
This comment has been minimized.
cbbaaa1989
commented
Jul 4, 2016
prerender get response of 304,how can i fix it? |
This comment has been minimized.
This comment has been minimized.
apaatsio
commented
Sep 2, 2016
I hit a problem with a redirect loop. This can happen if you have a redirect from http to https. To detect this you can curl your site To fix this you need to update this row This situation can occur when you have a setup where you have one server/proxy handling the SSL termination and http->https redirect and a separate web server that is just an http server running nginx with prerender configuration. |
This comment has been minimized.
This comment has been minimized.
yogeshsajanikar
commented
Sep 14, 2016
How can this be mixed with open_file_cache? |
This comment has been minimized.
This comment has been minimized.
jsonmaur
commented
Oct 2, 2016
Confused about why we need a long blacklist of extensions. Wouldn't we just want a whitelist instead? Such as |
This comment has been minimized.
This comment has been minimized.
jdkealy
commented
Oct 5, 2016
•
how would this work if your nginx has an upstream proxy?
|
This comment has been minimized.
This comment has been minimized.
I don't get any notifications when people comment on this thread so please email me at support@prerender.io if anyone has any questions. We're happy to help you get set up. |
This comment has been minimized.
This comment has been minimized.
chatoo2412
commented
Nov 16, 2016
•
I wonder why |
This comment has been minimized.
This comment has been minimized.
scne
commented
Nov 26, 2016
If chain of |
This comment has been minimized.
This comment has been minimized.
scne
commented
Nov 27, 2016
Please add |
This comment has been minimized.
This comment has been minimized.
simonepri
commented
Dec 17, 2016
@thoop What do you think about the improvment propsed by @rogierslag ? Also i think that these: |
This comment has been minimized.
This comment has been minimized.
zwolej
commented
Jan 6, 2017
for people that trying to use this config on own prerender servers, you must just replace:
|
This comment has been minimized.
This comment has been minimized.
Sureshkumars
commented
Jan 28, 2017
•
@thoop
} |
This comment has been minimized.
This comment has been minimized.
bdombro
commented
Feb 10, 2017
I had to add googlebot for google's webmaster tool's "Fetch as Google" to work. https://www.google.com/webmasters/tools/googlebot-fetch-details |
This comment has been minimized.
This comment has been minimized.
hetdev
commented
Feb 21, 2017
Hi |
This comment has been minimized.
This comment has been minimized.
jeerbl
commented
Mar 1, 2017
•
This does not include Google +. As stated in the Google+ docs, the user agent contains:
which should be added to the list of possible user agents. |
This comment has been minimized.
This comment has been minimized.
ietuday
commented
Apr 19, 2017
Hi Guys, I was running four node servers using Configuring multiple applications on same port using nginx reverse proxy based on request urls its working but the bundles files cannot loaded. I am stuck. Please Help on Browser Console Failed to load resource: the server responded with a status of 404 (Not Found) |
This comment has been minimized.
This comment has been minimized.
I don't get any notifications when people comment on this thread so please email me at support@prerender.io if anyone has any questions. We're happy to help you get set up. |
This comment has been minimized.
This comment has been minimized.
regilero
commented
May 18, 2017
Some modifications, especially |
This comment has been minimized.
This comment has been minimized.
fedealconada
commented
Jun 28, 2017
•
is it possible to avoid prerendering some specific files? as I have a bower dependency that, when is prerendered, is adding an html tag that is causing the bot to take an incorrect Title name of the page...(ngnix config) |
This comment has been minimized.
This comment has been minimized.
azulkipli
commented
Jul 28, 2017
•
This comment has been minimized.
This comment has been minimized.
jeerbl
commented
Aug 4, 2017
As stated in the Yahoo! Help, the Yahoo bot identifies itself as:
Therefore adding |
This comment has been minimized.
This comment has been minimized.
fernandosanchezjr
commented
Aug 10, 2017
How would one use uWSGI with this config? We have a django site with angular front-end, and we would like to cache static files. |
This comment has been minimized.
This comment has been minimized.
scne
commented
Aug 24, 2017
I've trouble with whatsapp, seems that not full url is catch: I see log of nginx and when whatsapp's crawler explore my url this match only |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Sep 24, 2017
Hi everyone, I have an issue with my config, google crowl only my home page, even i suggest the other pages it's shows the home page content. Also can i put a config for node.js and another in nginx in the same time ? Many thanks in advance !
} |
This comment has been minimized.
This comment has been minimized.
Startouf
commented
Nov 29, 2017
Seems like this guy has a working approach using |
This comment has been minimized.
This comment has been minimized.
blackandred
commented
Dec 9, 2017
Code review failed, one time you put boolean in a variable, other time a string |
This comment has been minimized.
This comment has been minimized.
sam-untapt
commented
Dec 15, 2017
Sorry, n00b question. If I have configured my nginx.conf file correctly, when I hit https://service.prerender.io/https://myurl (which displays correctly), will this request show up on https://prerender.io? Currently I see the message "We haven't seen a request with your Prerender token yet." Thanks. |
This comment has been minimized.
This comment has been minimized.
thucnd
commented
Jan 29, 2018
•
Hi All,
|
This comment has been minimized.
This comment has been minimized.
mdeolmos
commented
Feb 5, 2018
Is there a way to set YOUR_TOKEN as an environment variable? |
This comment has been minimized.
This comment has been minimized.
IuryAlves
commented
Feb 6, 2018
You can do that using the lua-nginx-module |
This comment has been minimized.
This comment has been minimized.
closca
commented
Mar 9, 2018
Hello, i see that SkypeUriPreview and other new crawlers are missing (eg: WhatsApp) |
This comment has been minimized.
This comment has been minimized.
courtneyschwartz
commented
Mar 23, 2018
@sam-untapt Change your browser's user agent string (there are extensions for this) to look like a bot, then try the request. The pre-render shouldn't be hit if you're browsing normally. |
This comment has been minimized.
This comment has been minimized.
courtneyschwartz
commented
Mar 23, 2018
•
Working w/ a hosting provider, we couldn't use the above nginx.conf as is. (Restricted to what we can add within a single location block.) Below is the alternative config for inside a
|
This comment has been minimized.
This comment has been minimized.
jordanandres1
commented
Apr 7, 2018
I tried with the follow configuration and it didn't work.
|
This comment has been minimized.
This comment has been minimized.
RicoFactset
commented
May 8, 2018
Hey, I just read ajax-crawling and the usage of escaped_fragment is deprecated: https://developers.google.com/webmasters/ajax-crawling/docs/specification I guess |
This comment has been minimized.
This comment has been minimized.
celiovasconcelos
commented
May 13, 2018
•
Why try_files? Using
The official snippet seems not to do that. Am I crazy? |
This comment has been minimized.
This comment has been minimized.
jindrichbartek
commented
Jul 7, 2018
•
From "Deprecating our AJAX crawling scheme" (https://webmasters.googleblog.com/2017/12/rendering-ajax-crawling-pages.html)
So what do you think? Did you guys do some tests with "escaped_fragment" condition? |
This comment has been minimized.
This comment has been minimized.
mindtonic
commented
Jul 11, 2018
•
+1 to @jeerbl for pointing out that this config file is missing Google Plus. The Google+ Snippet Fetcher User Agent is:
For my part I added it to my config file UserAgent regex as Also, if you want to use the Structured Data Testing Tool, you will want to add Final example:
|
This comment has been minimized.
This comment has been minimized.
maxklenk
commented
Jul 20, 2018
Please add |
This comment has been minimized.
This comment has been minimized.
ddereszewski
commented
Jul 27, 2018
if we use proxy for caching requests can we use proxy_cache_bypass based on user agent? The goal is that any refresh request made by prerender always takes latest version of the content. Any suggestions? |
This comment has been minimized.
This comment has been minimized.
theChumpus
commented
Aug 22, 2018
•
You may have already found a solution for this but I just ran into this too. Rather than cache based on user-agent (which didn't seem efficient as there are lots of possible UA strings), I use the same Edit: |
This comment has been minimized.
This comment has been minimized.
dimitarz
commented
Dec 24, 2018
I tried adding this config and it didn't work right away because of my try_files:
I ended up having to remove $uri/ =404 and it worked, but now non-existent pages returned index.html! I don't want this behavior, and narrowed it down to removing that pesky
Why does this even exist here? Totally confusing. |
This comment has been minimized.
This comment has been minimized.
ethicalmohit
commented
Feb 6, 2019
I am getting Error: Command:
|
This comment has been minimized.
pvolyntsev commentedFeb 15, 2014
condition
incorrect because
mean "string consist of one any character THEN 'ico'" that matched the URL /icon/ and it is wrong
I suggest this one:
that means "string is finished dot and one of strings 'js' OR 'css' etc"