Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Multiple Requests with Request (Node.js)
var request = require('request')
/**
* Handle multiple requests at once
* @param urls [array]
* @param callback [function]
* @requires request module for node ( https://github.com/mikeal/request )
*/
var __request = function (urls, callback) {
'use strict';
var results = {}, t = urls.length, c = 0,
handler = function (error, response, body) {
var url = response.request.uri.href;
results[url] = { error: error, response: response, body: body };
if (++c === urls.length) { callback(results); }
};
while (t--) { request(urls[t], handler); }
};
/*
* How to use __request()
* step by step
*/
// create an array of URLs
var urls = ["http://www.example.com/firts", "http://www.example.com/second", "http://www.example.com/third"];
// execute the request
// and assign a callback
__request(urls, function(responses) {
// When all the requests are ready
// this callback will be called
// you will get an argument, is
// a map with all the responses
// of the request you made,
// something like this:
/*
responses = {
"http://www.example.com/firts": {
error: [object Object],
response: [object Object],
body: [object Object]
},
"http://www.example.com/second": {
error: [object Object],
response: [object Object],
body: [object Object]
},
"http://www.example.com/third": {
error: [object Object],
response: [object Object],
body: [object Object]
}
}
*/
// Acces to a response:
// direct reference
var first_response = response["http://www.example.com/first"];
// check for errors of the first response
console.log(first_response.error);
// access to the body of the first response
console.log(first_response.body);
// also you can reuse the reference on the original array
var first_response = response[urls[0]];
// Iterate responses:
// You can simply iterate all responses
// to find errors or process the response
var url, response;
for (url in responses) {
// reference to the response object
response = responses[url];
// find errors
if (response.error) {
console.log("Error", url, response.error);
return;
}
// render body
if (response.body) {
console.log("Render", url, response.body);
}
}
});
@mikeal

This comment has been minimized.

Copy link

@mikeal mikeal commented Mar 9, 2012

this works, i wouldn't use arguments.callee tho.

you might want to check out async or another flow control library, that have more generic APIs for handling this kind of stuff, but if this works for you it looks fine.

@natos

This comment has been minimized.

Copy link
Owner Author

@natos natos commented Mar 12, 2012

Thanks for the feedback, I've made some changes according to JSLint parser and your suggestions.

@sushidub

This comment has been minimized.

Copy link

@sushidub sushidub commented Jun 20, 2012

Great Gist. Is there any chance you could show an example of how this looks in an actual request? Having trouble figuring out the syntax and/or format. Right now I'm storing my urls in an array like this:

var URLStoRequest = [http://www.foo1.com, http://www.foo2.com]
//executing
request({
uri:URLStoRequest
},
callback function

@natos

This comment has been minimized.

Copy link
Owner Author

@natos natos commented Jun 20, 2012

@sushidub I've created an example file, check it out and let me know if you need any help.

@sushidub

This comment has been minimized.

Copy link

@sushidub sushidub commented Jun 20, 2012

sweet. I'll give it a shot

@sushidub

This comment has been minimized.

Copy link

@sushidub sushidub commented Jun 20, 2012

Natos - It works! I understood what was happening in the Gist but the example clears up a few ambiguities. Very helpful. Thanks for the quick reply.

@natos

This comment has been minimized.

Copy link
Owner Author

@natos natos commented Jun 20, 2012

Cool, let me know if you need more hints.

@ghprod

This comment has been minimized.

Copy link

@ghprod ghprod commented Jan 1, 2014

I've got something error like this, any idea?

var first_response = response["http://cookingdiets.com"];
                                 ^
TypeError: Cannot read property 'http://cookingdiets.com' of undefined
@ghprod

This comment has been minimized.

Copy link

@ghprod ghprod commented Jan 1, 2014

ah, seems like in your example gist #43 you need to replace response with responses :)

@aniketrk

This comment has been minimized.

Copy link

@aniketrk aniketrk commented Aug 10, 2014

What if the url is not reachable or if the url doesn't exists. Then how will we come to know that which url was not reachable because in that that case we get response as undefined.

@shishuo365

This comment has been minimized.

Copy link

@shishuo365 shishuo365 commented Nov 30, 2015

When I use request 2.66, it can't find response.request

@vaibhavpacharya

This comment has been minimized.

Copy link

@vaibhavpacharya vaibhavpacharya commented Aug 22, 2017

Unfortunately doesn't work. My code is this -

      var links              = [urls.singleMovieInfo +body.results[0].id+'?api_key='+apiKey,
                                urls.singleMovieInfo +body.results[0].id+'/credits?api_key='+apiKey,
                                urls.nowPlaying +apiKey];
        request(links, function(err, responses){
            if(err){
              console.log(err);
            } else {
              console.log(responses);

It gives me the Error: options.uri is a required argument.

@serganus

This comment has been minimized.

Copy link

@serganus serganus commented Nov 17, 2017

Cool @natos

I re-wrote a promised version, check it out here: https://gist.github.com/serganus/ebea6717ce92df88061b0b96bef7c63e

@alibakhtiar

This comment has been minimized.

Copy link

@alibakhtiar alibakhtiar commented May 17, 2018

Nice work 👍

@dilippanwar1

This comment has been minimized.

Copy link

@dilippanwar1 dilippanwar1 commented Jun 4, 2018

Hi,

I have multiple GET/POST API's and in that i have to set the body, header content-type and then I need to post it with http method.
Can you please share an example how this could be done.
var options = { method: 'POST',
url: 'http://sdsdksdlas.com/api',
headers:
{
'content-type': 'application/json' },
body:
{ header: 'networork=verizon,asnum=1sfa000+11dasd6,location_id=0',
ip: 'NULL' },
json: true };

$http(options, function (error, response, body) {
console.log('Response code: ' + response.statusCode);

        if (!error && response.statusCode == 200) {
          console.log('GET authentication was successful');                 
          console.log(body);
        }
        else
        {
            console.log('GET failed');
            throw new Error(response.statusCode + ' response code received');
        }

});

@kaputerkeun

This comment has been minimized.

Copy link

@kaputerkeun kaputerkeun commented Jan 23, 2020

how if the process is post and contain json file?

@PerseusArkouda

This comment has been minimized.

Copy link

@PerseusArkouda PerseusArkouda commented Jan 10, 2021

how if the process is post and contain json file?

Change:
while (t--) { request(urls[t], handler); }
to
while (t--) { request(urls[t], { json: true }, handler); }

and:

if (response.body) {
                        console.log("Render", url, response.body);
 }

to:

if (response.body) {
                       Object.keys(response.body).forEach(e => result=`${response.body[e]}`);
                        console.log(result);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment