gabrielf/README.md

## README.md

      
    Raw
  

              README.md
            
          
    This script will use the dns.lookup function, which is used by node internally when making http requests, to lookup an IP and a hostname 3 times. A healthy system should give output similar to:
lookup of 93.184.216.119: 0ms
lookup of 93.184.216.119: 0ms
lookup of 93.184.216.119: 0ms
lookup of example.com: 6ms
lookup of example.com: 1ms
lookup of example.com: 0ms

Note that the first hostname lookup took longer than the following because of caching in some DNS. An unhealthy system might give you output like this:
lookup of 93.184.216.119: 0ms
lookup of 93.184.216.119: 0ms
lookup of 93.184.216.119: 0ms
lookup of weird.example.com: 84ms
lookup of weird.example.com: 80ms
lookup of weird.example.com: 82ms

Which will absolutely kill your performance. If you instead make many parallel dns lookups in the second situation you will see this:
lookup of weird.example.com: 96ms
lookup of weird.example.com: 98ms
lookup of weird.example.com: 98ms
lookup of weird.example.com: 98ms
lookup of weird.example.com: 190ms
lookup of weird.example.com: 190ms
lookup of weird.example.com: 190ms
lookup of weird.example.com: 190ms
lookup of weird.example.com: 286ms
lookup of weird.example.com: 287ms
lookup of weird.example.com: 287ms
lookup of weird.example.com: 287ms
...

What you see is (I think) Node's internal thread pool making DNS lookups. It seems there are four threads working on this in the above test. Imagine that you want to make 100 requests. Then most of the requests would just wait for the hostname to be resolved. The last request would have to wait 100/4 * dns lookup time ≈ 2400ms.
If you use the request library https://github.com/mikeal/request and set a timeout that is lower than this value then you will get ETIMEOUT even though the actual request has not even been sent yet, it has just waited in the queue to resolve the hostname. But the pain does not stop there. Lets say you try to make 500 requests to weird.example.com, each of which will trigger a slow dns lookup, then even if most of the requests fail with ETIMEOUT the dns lookup has still been queued which means that a single request made afterwards will have to wait for all of the previous dns lookups to have completed (or it might just timeout and just add another entry to the queue).

  
## nodejs-dns-vs-ip-lookup-test.js
// This gist is helpful in determining if the dns resolution is the bottleneck when making
// outbound http request
var dns = require('dns');

// enter the hostname and it's ip of the service you are making requests against below
var host = '<enter dns name here>';
var iphost = '<enter IP here>';

function testLookupSpeed(host, label, count) {
  console.time(label);
  dns.lookup(host, function(err, address, family) {
    if (err) throw err;
    console.timeEnd(label);
    if (--count) {
      testLookupSpeed(host, label, count);
    }
  });
}

testLookupSpeed(iphost, 'lookup of ' + iphost, 3);
testLookupSpeed(host, 'lookup of ' + host, 3);
	// This gist is helpful in determining if the dns resolution is the bottleneck when making
	// outbound http request
	var dns = require('dns');

	// enter the hostname and it's ip of the service you are making requests against below
	var host = '<enter dns name here>';
	var iphost = '<enter IP here>';

	function testLookupSpeed(host, label, count) {
	console.time(label);
	dns.lookup(host, function(err, address, family) {
	if (err) throw err;
	console.timeEnd(label);
	if (--count) {
	testLookupSpeed(host, label, count);
	}
	});
	}

	testLookupSpeed(iphost, 'lookup of ' + iphost, 3);
	testLookupSpeed(host, 'lookup of ' + host, 3);