DNS look up , example of Non-Blocking function of JS .

gistfile1.js

var SiteArray = ['google.com','webonise.com','facebook.com','techcrunch.com','indiatimes.com'];

var dns = require('dns');

for(var i = 0 ; i < SiteArray.length ; i++){
	dns.lookup(SiteArray[i],function(err,ip){
		if (err) return handleError(err);
		console.log( " %s resolved to %s ", SiteArray[i],ip );
	});
}

@Vishal , you solutions works out of box . But what I was trying to achive and understand is why the code I wrote is not working . In same code block , when first time SiteArray[i] do it right , and inside callback function it would not work.

And as per the scope of javascript, inner block can access outer block variables , but this dosnt work in that case too .

@yeshprit , I was under impression that callback get called once function execution is finisehd , but you are saying that function and call back get called asynchronous ?

@shardul, That's right that callback get called once function execution is finished.

JavaScript follow many programming paradigm and callback (passing function as argument to another function) is coming from Functional Programming paradigm. There are two types of callbacks: blocking callbacks (also known as synchronous callbacks or just callbacks) and deferred callbacks (also known as asynchronous callbacks). So when there is any I/O(AJAX etc.) or event handling associate with code we need to have deferred callbacks, because these will not block rest of script, so execution of function where we have passed asynchronous callbacks, will wait until its not received certain input. So its seems that following function stop execution in each loop but actually it doesn't.

dns.lookup(SiteArray[i],function(err,ip){
  if (err) return handleError(err);
     console.log( " %s resolved to %s ", SiteArray[i],ip );
  });  

Function remain in memory because its is waiting for DNS lookup which is I/O (Network opreation), where as see below code :-

var SiteArray = ['google.com','webonise.com','facebook.com','techcrunch.com','indiatimes.com'];

for(var i = 0 ; i < SiteArray.length ; i++){
  callbackDemo(SiteArray[i], function(i){
    console.log(i)
  });
}


function callbackDemo(value, callback) {
  if(value) {
    callback(value)
  }
}

above is code example of synchronous callbacks its not waiting for any blocking operation, you'll see expected output because this works sequentially.

@shardulmohite: Hmm, this is problem with asynchronous calls made while iterating over something -- in such case the calls happen when the loop completes and all the subsequent calls are able to capture only the last element in the iteration.

Above code didn't work because dns.lookup always needs the variable i, and as its accessed by reference (in case of asynchronous callbacks, this is a must situation), its value is set to the last assigned value to it (that is last value in the iteration). So, instead sending same reference of i, as @yeshpritweb suggested above, either --

• we can invoke an anonymous function within a closure which inside gets the different reference in the arguments while executing
• or call another named function, which similarly gets the new reference every time while executing, hence the it will work as intended. See following modified code:

var SiteArray = ['google.com','webonise.com','facebook.com','techcrunch.com','indiatimes.com'];

var dns = require('dns');

function resolve ( site ) {
  dns.lookup( site, function( err, ip ){
    if ( err ) return handleError( err );
        console.log( " %s resolved to %s ", site, ip );
    });     
}

for(var i = 0 ; i < SiteArray.length ; i++){
  resolve( SiteArray[i] );
}

So as pointed out by @yeshpritweb -- this is common issue with the asynchronous (or callback) functions. Asynchronous AJAX, or a file read operation call made similarly will have same consequences.

Thanks @Vishal ,

here is but what I think is going on in there ,

for(var i = 0 ; i < SiteArray.length ; i++){
    dns.lookup(SiteArray[i],function(err,ip){
        if (err) return handleError(err);
        console.log( " %s resolved to %s ", SiteArray[i],ip );
    });
}

In above code, For Loop is sequential , but inside every for loop , dns.lookup get called , now as Javascript is asynchronous , for loop continue to execute sequentially and 5 function calls are made almost immediately ,

so

dns.lookup('google.com', .... ) -> i=0
dns.loopup('webonise.com',...) -> i=1
dns.lookup('facebook.com',...) -> i = 2
dns.lookup('techcrunch.com',...) -> i=3
dns.lookup('indiatimes.com',....) -> i=4

All the above 5 function calls are made and because of For loop they are made in sequential manner, dns.lookup is I/O call as mentioned by @yashprit , they are going to take some time , and then depending on the speed of site and how faster the respective server respond to resolve IP address , callbacks will start getting called .

We are printing IP address in Callback function of dns.lookup.

Callbacks start getting called when I/O request of reverse lookup and Function execution is finished , and this is where two things happens .

1. most of these callbacks are starting after For loop execution is finished, and at the end of for loop execution is finished value of i is 5 (as yashprit said earlier ) .
2. callbacks are not getting called in sequential (as they should not anyway ) , if you see the printout put , you will find something like below .

 facebook.com resolved to 173.252.110.27
 techcrunch.com resolved to 66.155.9.244
 google.com resolved to 173.194.36.9
 webonise.com resolved to 198.61.175.81
 indiatimes.com resolved to 223.165.27.13 

So if array might be have been larger with more than 5 values, lets say around 100 values, would we still get the same error with my code ?
Answer is "Might not" , as in this case , before the for loop execution finished, some of the callbacks would have started getting called and whatever i value at that time would have been printed . But the code would have not printed exactly what was expected also.

Vishal,

following lines from your comment
Above code didn't work because dns.lookup always needs the variable i, and as its accessed by reference (in case of asynchronous callbacks, this is a must situation)

In my understating is wrong interpretation of above code because JavaScript doesn't support Passing by reference and even if it has supported that than also in above case its pass by value not pass by reference. In case of JavaScript is always pass by value, but when a variable refers to an object (including arrays), the value is a reference to the object.

So for only understanding purpose we can say that When passing in a primitive type variable like a string or a number, the value is passed in by value otherwise pass by reference.

So in above case while iterating over array and using closure we are preserving value of i i.e. we are creating a closure and invoking it.The value i inside the body of the closure is being bound to the same instance for each closure. i.e. calling self executing anonymous function as pass by value for various i

It is similar to what you are doing in example that you have given, you are passing value SiteArray[i] as value, so even value of i is changing its not effecting called function, in case of reference it might have.

Both Point are same because both has different function for executing callback code only difference is named function and anonymous.

I think asynchronous callbacks are powerful tool, this makes NodeJS so popular and faster blocking I/O has bigger issue like accessing DB could stop script until its not received any input.

Shardul,

Regarding large element in array what could be solution, I think problem remain same, I've tested above code with 15648 elements I got same issue.

When I dig more into this we all know JavaScript is Single thread programming language, and async callback or non blocking I/O works on event rather than threads or process because threads/process carry a heavy memory cost.(as Threads and process are heavier or system might not support many that much threads, this was one of reason why Nginx(event-based) become so popular than Apache(Thread or process-based on configuration) ).

So what happening behind the scene is follows :-

--Whole code runs in the main thread and main thread register async callback to event loop.(An event loop is “an entity that handles and processes external events and converts them into callback invocations”. ) At an I/O call, your code saves the callback and returns control to the node.js runtime environment. So for below line

for(var i = 0 ; i < SiteArray.length ; i++){
    dns.lookup(SiteArray[i],function(err,ip){
        if (err) return handleError(err);
        console.log( " %s resolved to %s ", SiteArray[i],ip );
    });
}

dns.lookup is register with event-loop, and execution return back to NodeJS runtime environment, this repeat with all element of array.

--After executing for loop NodeJS runtime proceed with executing other statement of function sequentially. so if you have written code below loop that has another loop that prints array you'll see that value before you started seeing callbacks console.log.

--After executing for loop or other part of code, and if any event occurred then you'll see execution of callbacks but while executing for loops any event occur on event-loop than callback will not be trigger because of Single thread and that thread is busy doing other job.

👍