Instantly share code, notes, and snippets.

Embed
What would you like to do?
Lazy-rendering in PhantomJS
// This example shows how to render pages that perform AJAX calls
// upon page load.
//
// Instead of waiting a fixed amount of time before doing the render,
// we are keeping track of every resource that is loaded.
//
// Once all resources are loaded, we wait a small amount of time
// (resourceWait) in case these resources load other resources.
//
// The page is rendered after a maximum amount of time (maxRenderTime)
// or if no new resources are loaded.
var resourceWait = 300,
maxRenderWait = 10000,
url = 'https://twitter.com/#!/nodejs';
var page = require('webpage').create(),
count = 0,
forcedRenderTimeout,
renderTimeout;
page.viewportSize = { width: 1280, height : 1024 };
function doRender() {
page.render('twitter.png');
phantom.exit();
}
page.onResourceRequested = function (req) {
count += 1;
console.log('> ' + req.id + ' - ' + req.url);
clearTimeout(renderTimeout);
};
page.onResourceReceived = function (res) {
if (!res.stage || res.stage === 'end') {
count -= 1;
console.log(res.id + ' ' + res.status + ' - ' + res.url);
if (count === 0) {
renderTimeout = setTimeout(doRender, resourceWait);
}
}
};
page.open(url, function (status) {
if (status !== "success") {
console.log('Unable to load url');
phantom.exit();
} else {
forcedRenderTimeout = setTimeout(function () {
console.log(count);
doRender();
}, maxRenderWait);
}
});
@mpdmanash

This comment has been minimized.

Show comment
Hide comment
@mpdmanash

mpdmanash Jan 24, 2014

Thanks a lot.
I made an account at GitHub to thank you.

mpdmanash commented Jan 24, 2014

Thanks a lot.
I made an account at GitHub to thank you.

@dorajistyle

This comment has been minimized.

Show comment
Hide comment
@dorajistyle

dorajistyle Feb 6, 2014

Brilliant!
Thanks a lot.

dorajistyle commented Feb 6, 2014

Brilliant!
Thanks a lot.

@kimmobrunfeldt

This comment has been minimized.

Show comment
Hide comment
@kimmobrunfeldt

kimmobrunfeldt Mar 7, 2014

I made a node module which was inspired by this, check it out https://github.com/kimmobrunfeldt/url-to-image !

kimmobrunfeldt commented Mar 7, 2014

I made a node module which was inspired by this, check it out https://github.com/kimmobrunfeldt/url-to-image !

@frankred

This comment has been minimized.

Show comment
Hide comment
@frankred

frankred May 5, 2014

Thank you, I do not use this for making a screenshot, but to give JS AJAX pages some time to build up the page, after that I can scrape usefull information...

frankred commented May 5, 2014

Thank you, I do not use this for making a screenshot, but to give JS AJAX pages some time to build up the page, after that I can scrape usefull information...

@raviksingh09

This comment has been minimized.

Show comment
Hide comment
@raviksingh09

raviksingh09 Aug 9, 2014

Can you please advice why does this script fail for "http://www.theveteranssite.com" ?

raviksingh09 commented Aug 9, 2014

Can you please advice why does this script fail for "http://www.theveteranssite.com" ?

@dirkschneemann

This comment has been minimized.

Show comment
Hide comment
@dirkschneemann

dirkschneemann Jan 21, 2015

Hooray, this is the only code I found so far on this topic which is really working as intended, thank you!

dirkschneemann commented Jan 21, 2015

Hooray, this is the only code I found so far on this topic which is really working as intended, thank you!

@nedrocks

This comment has been minimized.

Show comment
Hide comment
@nedrocks

nedrocks Aug 24, 2015

This is fantastic. Thank you very very much for this snippet!

nedrocks commented Aug 24, 2015

This is fantastic. Thank you very very much for this snippet!

@goudgoud69

This comment has been minimized.

Show comment
Hide comment
@goudgoud69

goudgoud69 Aug 29, 2016

Thank you ! It' work fine and it's so simplier !

goudgoud69 commented Aug 29, 2016

Thank you ! It' work fine and it's so simplier !

@sramzan

This comment has been minimized.

Show comment
Hide comment
@sramzan

sramzan Oct 3, 2016

Thank you!... I've been having an issue where phantom would not wait for all resources to load. My initial method of static wait times wasn't working, so this was an awesome solution.

sramzan commented Oct 3, 2016

Thank you!... I've been having an issue where phantom would not wait for all resources to load. My initial method of static wait times wasn't working, so this was an awesome solution.

@dm-wyncode

This comment has been minimized.

Show comment
Hide comment
@dm-wyncode

dm-wyncode Nov 5, 2016

Thanks for this. It solved a lot of issues for some code I was writing.

dm-wyncode commented Nov 5, 2016

Thanks for this. It solved a lot of issues for some code I was writing.

@dvjunior

This comment has been minimized.

Show comment
Hide comment
@dvjunior

dvjunior Apr 7, 2017

How can I set proxy authentication? I tried this: phantomjs --proxy-auth=user:pass myTwitter.js but response is 401.
I have a script for screen capture and I and I was able to authenticate with the same way up.

ex: phantomjs --proxy-auth=user:pass --ignore-ssl-errors=true myScript.js http://urlforexample.com output.png timeout 1400(px) 700(px)

myScript.js

dvjunior commented Apr 7, 2017

How can I set proxy authentication? I tried this: phantomjs --proxy-auth=user:pass myTwitter.js but response is 401.
I have a script for screen capture and I and I was able to authenticate with the same way up.

ex: phantomjs --proxy-auth=user:pass --ignore-ssl-errors=true myScript.js http://urlforexample.com output.png timeout 1400(px) 700(px)

myScript.js

@dvjunior

This comment has been minimized.

Show comment
Hide comment
@dvjunior

dvjunior Apr 7, 2017

I was able to solve with page.settings.userName and page.settings.password, but this script twitter.js did not solve my problem, was animation in page, the data is loaded via ajax, in my case, phantomjs render the animation.

dvjunior commented Apr 7, 2017

I was able to solve with page.settings.userName and page.settings.password, but this script twitter.js did not solve my problem, was animation in page, the data is loaded via ajax, in my case, phantomjs render the animation.

@oeurnravuth

This comment has been minimized.

Show comment
Hide comment
@oeurnravuth

oeurnravuth May 27, 2017

Thank you so much for share this!

oeurnravuth commented May 27, 2017

Thank you so much for share this!

@bolsunovskyi

This comment has been minimized.

Show comment
Hide comment
@bolsunovskyi

bolsunovskyi Aug 28, 2017

Thanks, it helps!!!

bolsunovskyi commented Aug 28, 2017

Thanks, it helps!!!

@peterruijter

This comment has been minimized.

Show comment
Hide comment
@peterruijter

peterruijter Oct 18, 2017

Nice script. A small side note; why do you declare the variable forceRenderTimeout? This one is never used and setTimeout can be called without an assignment to a variable.

peterruijter commented Oct 18, 2017

Nice script. A small side note; why do you declare the variable forceRenderTimeout? This one is never used and setTimeout can be called without an assignment to a variable.

@http91

This comment has been minimized.

Show comment
Hide comment
@http91

http91 Nov 17, 2017

It worked great!
Thanks a lot!

http91 commented Nov 17, 2017

It worked great!
Thanks a lot!

@opicron

This comment has been minimized.

Show comment
Hide comment
@opicron

opicron May 12, 2018

This is actually doing nothing:

renderTimeout = setTimeout(doRender, resourceWait);

Increase the resourceWait and set to:

renderTimeout = setTimeout(function () {
                    //console.log(count);
                    doRender();
                }, resourceWait);

opicron commented May 12, 2018

This is actually doing nothing:

renderTimeout = setTimeout(doRender, resourceWait);

Increase the resourceWait and set to:

renderTimeout = setTimeout(function () {
                    //console.log(count);
                    doRender();
                }, resourceWait);
@mmehedin

This comment has been minimized.

Show comment
Hide comment
@mmehedin

mmehedin Jun 13, 2018

Excellent. Thank you!

It works with everything I tried except this:
https://myaccount.nytimes.com/auth/login

I do not get the rendering of the input for username, password and the submit button.
Any ideas why? Really thankful for your code.

mmehedin commented Jun 13, 2018

Excellent. Thank you!

It works with everything I tried except this:
https://myaccount.nytimes.com/auth/login

I do not get the rendering of the input for username, password and the submit button.
Any ideas why? Really thankful for your code.

@presiyanPeykov

This comment has been minimized.

Show comment
Hide comment
@presiyanPeykov

presiyanPeykov Oct 13, 2018

Works like a charm .
Thank you .

presiyanPeykov commented Oct 13, 2018

Works like a charm .
Thank you .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment