https://github.com/nightwatchjs/nightwatch/wiki/Understanding-the-Command-Queue
When Nightwatch runs a test, it processes its commands in a list known as the command queue. This list manages the asynchronous execution of the commands defined in that test.
As a queue, the command queue generally follows the rule of first in, first out (FIFO). The first command you call in a test is the first executed when the test runs. The second command is called next, followed by the next command up until the last command added, which becomes the last command executed.
The command API in Nightwatch - accessible via the object passed into test cases, usually called "client" or "browser" - consists of a collection of methods that are used to construct the command queue. When you call a Nightwatch command such as click(), you're not sending the Selenium client the command to immediately click something, at least not right away. That method instead adds a "click" command to the command queue. After the test case function has resolved, something it does synchronously (commands are non-blocking), it traverses the command queue as defined by the Nightwatch commands you called, running through the queue executing each command in it asynchronously.
Consider the following test:
// test with 'click button' test case
module.exports = {
'click button': function (browser) {
browser
.url('example.com')
.click('button')
.end();
}
};Here, the 'click button' test case calls three methods from the browser object: url(), click(), and end(). When Nightwatch runs this test, it calls the 'click button' function which, in turn, calls each of these methods. These methods run synchronously, one directly and immediately after the other, and populates the command queue without performing any Selenium tasks. The result is a queue that looks something like:
// 'click button' test case command queue
[
{command: 'url', args: ['example.com']},
{command: 'click', args: ['button']},
{command: 'end', args: []}
]Its only after the test case function resolves that the queue is traversed and the commands executed, either in the context of Selenium, or through whatever other implementation they may have.
Most commands in Nightwatch are Selenium commands, wrapping a single, or two or more Selenium requests. These are handled through the WebDriver API which consists of a series of HTTP endpoints to perfom actions on a WebDriver client, such as a Selenium Server instance. The HTTP requests made by Nightwatch are non-blocking with the results of the requests handled in callbacks. The click command, for example, in the deep internals of Nightwatch code is executed using something like:
// running the Selenium click command
http.request({
method: 'POST',
path: '/session/1/element/2/click'
}, function (response) {
// request complete
});The non-blocking, asynchronous nature of Selenium-based commands, or even other asynchronous commands like pause() (effectively a command wrapper for setTimeout()) means that for operations like this to run in sequence, they must wait for the completion of the previous. The susinct, chainable command API of Nightwatch doesn't force this requirement on you, instead handling it internally through the inner workings of the command queue. It acts as a cache saving off the operations you wish to perform until after the test function completes at which point, it runs them each in sequence asynchronously.
// command queue execution pseudo code
commandQueue = []
runTestCaseFunction(browser) // synchronous (adds to commandQueue)
for command of commandQueue {
await runCommand(command) // asynchronous (commands wait for previous to finish)
}Just about every native Nightwatch command supports a callback function to be called when the command has been executed within the command queue. These callbacks, as with the execution of the commands themselves within the queue, are called asynchronously.
// click command callback
browser.click('button', function (result) {
// when the queue has finished executing the click command
// where `result` is the response from the Selenium click operation
});The primary purpose for callbacks is to allow you to capture data from the test while its running, but they also provide an additional opportunity to add commands to the command queue. Unlike the command methods called directly within the test case function body, command methods called in a callback get added to the current location of the command queue within its traversal. Commands added in the click() callback above, for example, would be added directly after the click command's location within the queue rather than being added to the end (the end being the standard FIFO enqueue behavior seen with commands outside of callbacks). Consider a revised sample test:
// test with 'click button' test case
module.exports = {
'click button': function (browser) {
browser
.url('example.com')
.click('button', function (result) {
browser.pause(100);
})
.end();
}
};When run, initially the command queue starts as:
// 'click button' test case command queue after test case called
[
{command: 'url', args: ['example.com']},
{command: 'click', args: ['button']},
{command: 'end', args: []}
]Note that there is no pause yet, because the pause() command method doesn't get called until after the click command is executed when the queue is being processed. This is the state of the queue immediately after the test case function runs.
When the queue starts, it runs the url command, waits for it to finish, then runs the click command which then calls the callback. When the callback runs, it adds the pause command to the queue, giving us:
// 'click button' test case command queue after click callback called
[
{command: 'url', args: ['example.com']}, // executed
{command: 'click', args: ['button']}, // executed
{command: 'pause', args: [100]}, // <-- just added
{command: 'end', args: []}
]When the queue continues to the next step, it then runs the newly added pause command - as that is now the next command in the queue - before eventually running the final end command completing the test case.
One thing to realize is that though the pause command was added out of order, asynchronously, the order in which you read each of the commands in the source code is ultimately the same order they are executed when the command queue is traversed. The added order is url, click, end, pause; while the executed order - and the order read in the source - is url, click, pause, end. This makes it easy to look at source code and immediately get a general idea of the order of commands ultimately getting run.
As mentioned earlier, callbacks provide a way to get data from a running test such as values of textfields, attributes, etc. This data isn't available when the test function runs because the actual running of the test case (traversing of the command queue) doesn't start until after the test case function has already run to completion.
// capturing test data from callback
var text;
browser.getValue('#input', function (result) {
text = result.value; // captured during command queue execution
});Similar to how commands in callbacks aren't in the command queue right away, values captured this way also aren't available until the test is running. In a callback, all code directly in the test case function body has already resolved, and the only place any other code will run is in other callbacks. This is important to keep in mind because it can be easy to think this might work:
// incorrect usage of a callback value
var text;
browser
.getValue('#input', function (result) {
text = result.value;
})
.setValue('#output', text); // WRONG: text is undefinedThe problem here is that the setValue() call happens in the main test case function call, before the callback is called when text is still undefined. For setValue() to have the correct value for text, it must be called within, or some time after, the getText() callback:
// correct usage of a callback value
var text;
browser.getValue('#input', function (result) {
text = result.value;
browser.setValue('#output', text); // RIGHT
});Any additional dependencies on text may also go in the getValue() callback, and any command there may have their own callbacks which may include additional commands ultimately giving you a mess of nested callbacks.
// nested callbacks
var text, text2, text3; // ...
browser.getValue('#input', function (result) {
text = result.value;
browser.getValue('#' + text, function (result) {
text2 = result.value;
browser.getValue('#' + text2, function (result) {
text3 = result.value;
browser.getValue('#' + text3, function (result) {
// ...
});
});
});
});To counter this, you can use the perform() command.
Nightwatch's perform() command is effectively a no op command that exists only to provide a callback allowing you to run code within the context of the running command queue. Within the callback, you're ensured to have code which will run after any callbacks defined above it, even if nested.
// perform to help combat nested callbacks
var text, text2, text3;
browser
.getValue('#input', function (result) {
text = result.value;
browser.getValue('#' + text, function (result) {
text2 = result.value;
});
})
.perform(function () {
browser.getValue('#' + text2, function (result) {
text3 = result.value;
browser.getValue('#' + text3, function (result) {
// ...
});
});
});Let's rundown of how this set of commands would play out in a test case, starting from when the test case is first called:
// initial test case queue
[
{command: 'getValue', args: ['#input', callback]},
{command: 'perform', args: [callback]},
]
// queue after first getValue ('#input') callback
[
{command: 'getValue', args: ['#input', callback]}, // executed, text defined
{command: 'getValue', args: ['#text', callback]}, // <-- added
{command: 'perform', args: [callback]},
]
// queue after second getValue ('#text') callback
[
{command: 'getValue', args: ['#input', callback]}, // executed, text defined
{command: 'getValue', args: ['#text', callback]}, // executed, text2 defined
{command: 'perform', args: [callback]},
]
// second getValue callback adds no commands, perform is next
[
{command: 'getValue', args: ['#input', callback]}, // executed, text defined
{command: 'getValue', args: ['#text', callback]}, // executed, text2 defined
{command: 'perform', args: [callback]}, // executed
{command: 'getValue', args: ['#text2', callback]}, // <-- added
]
// final queue after third getValue ('#text2') callback
[
{command: 'getValue', args: ['#input', callback]}, // executed, text defined
{command: 'getValue', args: ['#text', callback]}, // executed, text2 defined
{command: 'perform', args: [callback]}, // executed
{command: 'getValue', args: ['#text2', callback]}, // executed, text3 defined
{command: 'getValue', args: ['#text3', callback]}, // <-- added
]Callbacks (inline), like the commands themselves, are executed in the same order they're read in the code. So even though the perform() command itself is added to the queue before the text2 defining getValue() call is, its callback doesn't execute until after text2 has been defined in that getValue()'s callback.