chidumennamdi/blog.md Secret

## blog.md

      
    Raw
  

              blog.md
            
          
    Nodejs Event Loop system: A hand's on approach - Part 1

Node.js - An Event-based system

Node.js works on event-driven approach. This means that its flow control is determined by events or changes in state. This is called Event Driven Programming.
The general implementation of it (Event Driven Programming) is to have a mechanism that listens for events and calls the callback function associated with the event when raised.
Despite being single-threaded (Node.js), it is very important it doesn't block during IO operations but handle them concurrently.
This is the first in the series of posts I'll be writing about Node.js event loop system.

Nodejs Event Loop system: A hand's on approach - Part 1
Nodejs Event Loop system: Best practices - Part 2

What does this series offer us?

A deep look and technical knowledge about Node.js event loop.
A deep deep dive into libuv, the non-blocking engine of Nodejs. We will see how it works and how it provides asynchrony to Node.js.
Ability to write fast, non-blocking and efficient code that handles asynchronous calls.
best practices that take your Node.js skills to the next level.

In this first post, We will look into the basic details of IO operations (Non-blocking and blocking), libuv, event loop, and Node.js, providing a hand's on an example on each of them.
Blocking I/O, Non-Blocking I/O


IO refers to interaction with the OS disk and network.

In blocking IO programming, a function call that accesses an IO in the OS blocks the execution of the thread and leaves the system resources idle, until the IO operation completes. This code. Let's look at a blocking code:
https://gist.github.com/aac802fa8c3c29a3774b9551cc4547a3
Now, most OS are multithreaded, accessing IO can become non-blocking.
In this non-blocking IO operation, the call to IO resource returns without waiting for it to complete. It constantly queries the kernel for the completion mode of the resource and whenever there is data available it returns.
https://gist.github.com/2a17ceb29f0db47e55475523172e286e
Event Demultiplexer

The above non-blocking IO implementation is not the ideal solution. Many modern OSes like Windows, Unix, BSD systems have their implementations of event demultiplexer to handle concurrent, non-blocking resources in an efficient way.

kqueue (BSD, OSX)
epoll (Linux)
event ports (Solaris, SunOS)
IOCP GetQueuedCompletionStatusEx (Windows)

Event demultiplexer is a synchronous non-blocking notification system.
The Reactor Pattern

Reactor pattern is a pattern of synchronous demultiplexing and invocation of events handler in the order they arrive.
This is what happens in an application that uses reactor pattern:


The application makes an IO request. This request is delegated to the Event Demultiplexer which sends it to the appropriate hardware.


When the IO operations complete, Event demultiplexer pushes the associated callback to the Event Queue.


The event queue loops through the list and executes them in the order they were inserted until the list exhausts.


When the event queue is empty, the loop will block on the Event Demultiplexer and triggers another cycle 1..


If the event queue and the Event Demultiplexer pending requests queue is empty the program exits.


Event Queue

Event queue is a data structure where callbacks associated with an async IO operation are enqueued to be executed sequentially by the Event Loop until the queue is empty.
Event queue is composed of different queues each associated with an IO event. In Node.js, there are:

timers queue: Expired timers and due interval callbacks are enqueued here.
poll queue: Completed IO events are enqueued here.
check queue: setImmediates callback are enqueued here.
pending queue: Completed/Errored IO events are here.

besides these four queues, there are other two queues in Node.js

nextTick queue: queued by process.nextTick()
microtask queue: queued by reslved Promises.

Will get into these queues in the later sections.
libuv: Non-Blocking IO Engine of Node.js


"Concurrency is a way to structure a thing so that you can, maybe, use parallelism to do a better job. But parallelism is not the goal of concurrency; concurrency's goal is a good structure." - R. Pike, Golang co-inventor

We saw earlier different API in different Operating Systems used for the Event Demultiplexer. To support Event Demultiplexer on diff systems, the Nodejs group developed a library to provide a high-level abstraction and make Node.js compatible with all the major platforms.
The library was called libuv

libuv is cross-platform support library which was originally written for NodeJS. It's designed around the event-driven asynchronous I/O model.


The library provides much more than a simple abstraction over different I/O polling mechanisms: handles and streams provide a high-level abstraction for sockets and other entities; cross-platform file I/O and threading functionality is also provided, amongst other things.

We now see that the reactor pattern are the building blocks of Node.js. Gathering it all together, Node.js becomes a collection of utilities:

v8: A powerful and high-performance JavaScript engine from Google.
libuv: Library for async IO operations.
zlib: for compression, http_parser: for HTTP requests and responses, openssl: for HTTPS security

From what we stated above, we see that libuv provides event loop for Node.js. Let's look at the Event loop deeper.
What is Event Loop?

Event loop is the system in Nodejs that allows it to perform non-blocking IO operations. Non-blocking IO operations, what does it mean?
Answer for all these lies at the heart of NodeJS, the Event Loop. Since most OS are multithreaded, they handle multiple operations or programs running parallel to each other. Nodejs delegates any IO operations to the OS, and waits for the completion status of the operation and executes its callback when due.
When does Event Loop run?
When a script is run in Node.js, like this:
https://gist.github.com/a8a703c614d0b6ece91b35d40819c6ff
Node.js initializes Event Loop, and processes the provided script, in our case, it is script.js. This script may make async calls via async APIs like:

fs.readFileAsync
setTimeout, setInternal
setImmediate, etc

These APIs enqueue tasks in the event queue, so after processing the script, Node.js calls the Event Loop.
Let's look at the phases in the Event Loop.
https://gist.github.com/8223d1dac305a8cd0493f2a686649260
Each box represents a phase in the Event loop. We have the timers phase, pending callbacks phase, idle, prepare phase, poll phase, check phase and close phase.
These phases have a FIFO policy. The first callback queued is first dequeued and executed.
When the event loop enters a phase, it iterates through the queue in the phase and proceeds to each the callbacks until the queue is exhausted or the max limit for the phase is reached. When either of these occurs the event loop proceeds to the next phase and does the same thing all over again.
The Event Loop exits when all the queues in the phases are all empty.

timers: This is scheduled by setTimeout, setInterval.
pending callbacks: Any completed,errored IO operation is queued here.
idle, prepare: These are used internal operations of Node.
poll: Node will block here for a specified amount of time to wait for any completed IO operation.
check: It executes callbacks scheduled by setImmediate.
close: It runs all callbacks queued by any closed IO operation, eg. *.on('close', callback), socket.destroy().

Let's get into more detail about the timer and poll phases and see their implementation in libuv. These two are the most important phases in the event loop, critical decisions are made in their phases.
Timers
This is the first phase in the event loop. This phase executes all callbacks scheduled by setInterval or setTimeout APIs. The timers provide a threshold after which the callback is executed.
https://gist.github.com/2d9db183320c7276b6f727ba290e7b85
Here, the callback function is executed after 100ms has elapsed. When the program is run node script.js, Node bootstraps itself and executes our provided script.js. This runs setTimeout which enqueues the callback. After executing the script, Node runs the event loop, which executes the callback function and we see setTimeout on the terminal.
Though, the callback is not immediately run. Event loop checks if the threshold set has elasped, if not, it runs other phases and cycles back for another pass tick.
Let's demonstrate with this example:
https://gist.github.com/de4bf5746cd02754c9edb6078da5212a
We kinda simulated the timers phase. I created a loop object, which contains everything needed for the event loop. That the same thing done on libuv there is a central uv_loop_t struct. It contains all the phases.
Our loop has a time property which holds the current time, this is updated at the start of a cycle. There is the timer_heap which holds the setTimeouts callbacks in an array. Though, this not how it is implemented in libuv. In libuv, timers are structured in a linked list in a heap_node struct:
https://gist.github.com/0586f14391e17ac7e52bf1945977489e
left points to its previous heap_node, right points to the next node inline. There is no pushing and popping.
But, our implementation provides the same results as above. Next, we have the enqueue property, which pushes a timer to the timer_heap array and sorts them with the lowest timer being first. min follows which is used to get the timer with the nearest threshold since the timer_heap is already sorted, it just returns the 0 index timer.
Next, we began implementing some functions, we would find in the libuv.
uv_timer_stop dequeues the nearest timer, uv_update_time updates the loop.time with the current time.
uv_run_timers is the function that runs the timers phase.
It begins with an infinite for-loop, inside the loop, it gets the nearest timer if there is no timer in the timer_heap it breaks off the loop. But if there is, it checks if the timer has expired, if yes, it dequeus the timer and runs its callback but if not, it breaks off.
Let's look at what libuv has:
https://gist.github.com/f609642d17cb6f938c48e9c7ca91bbcf
Our codes and implementation are similiar.
uv_loop_alive function simply checks if the timer_heap is empty. There is more to this, we only cheked the timer_heap b'cos it is the only phase we have. In libuv.
uv_run function is the Event Loop. It runs all the phases and surmises when to exit. In our implementation, it only runs the timer phase.
It runs all the phases in a while-loop, at each loop, it runs all phases and calls the uv_loop_alive to know whether to break off and exit.
In libuv, it's quite bigger. We'll come to it later.
Next, we monkey-patched the setTimeout function and set it up with our own implementation. It calls the loop's enqueue function on execution, to push its callback function cb and threshold ms to the timer_heap. The real setTimeout does the same thing though, in a different way but with the same result as ours.
We call the setTimeout function with different callbacks and timeouts and kick off the event loop with uv_run.
So we have simulated the timer phase in JavaScript. If we run our script, it will yield this:
https://gist.github.com/83e4eef06c8ccc552f222f8e302ccf2a
Same as it would, with the real setTimeout.
Calculation of time and threshold expiration vary on hardware and OS. So, this timer phase is a little bit buggy you can say.
Now, let's poll a bit :).
Poll
Polling watches for new connections, requests, data, etc. Generally, that's its job.
Looking at Node.js artcile about the poll phase, they wrote this:
https://gist.github.com/9b88e336fe1c5a76401e0096cf15acb6
But, looking at the libuv sources, the implemetations doesn't perfectly correspond to what is above.
The thing is that poling happens differently on different OSes.
Like we listed in the reactor section,

kqueue (BSD, OSX)
epoll (Linux)
event ports (Solaris, SunOS)
IOCP GetQueuedCompletionStatusEx (Windows)

Linux, BSD, OSX, Solaris, SunOS except Windows are built with Unix policy. They behave alike but have different implemetations. In libuv, the polling mechanism is carried out by the uv__poll function. libuv has two folders win and unix, win contains implemetation for Windows systems and unix for Unix-based OSes.
libuv detects which platform or OS it currently running on to know which folder to include.
Let's look at the Windows and Unix implementations:
Windows
https://gist.github.com/ecee6a0e065aca5fa1bc5f3909965142
The above code is event loop implementation for Windows. You see it clearly represents the event loop diagram we saw earlier.

uv__run_timers: runs the timers phase; checks for expired timers.
uv_process_reqs: processes pending requests.
uv_idle_invoke: run the idle phase.
uv_prepare_invoke: runs the prepare phase.
uv__poll: polls for completed IO operations.
uv_check_invoke: runs the callbacks set by setImmediate.
uv_process_endgames: runs the close phase.

In unix, it is different, uv_process_reqs is absent, it is replaced by uv__run_pending(loop). Here, in Win, there is no pending queue, pending requests from the uv__poll is processed there.
Let's look at uv__poll imple.:
// win/core.c
static void uv__poll(uv_loop_t* loop, DWORD timeout) {
 BOOL success;
 uv_req_t* req;
 OVERLAPPED_ENTRY overlappeds[128];
 ULONG count;
 ULONG i;
 int repeat;
 uint64_t timeout_time;

 timeout_time = loop->time + timeout;

 for (repeat = 0; ; repeat++) {
   success = GetQueuedCompletionStatusEx(loop->iocp,
                                         overlappeds,
                                         ARRAY_SIZE(overlappeds),
                                         &count,
                                         timeout,
                                         FALSE);

   if (success) {
     for (i = 0; i < count; i++) {
       if (overlappeds[i].lpOverlapped) {
         req = uv_overlapped_to_req(overlappeds[i].lpOverlapped);
         uv_insert_pending_req(loop, req);
       }
     }
     uv_update_time(loop);
   } else if (GetLastError() != WAIT_TIMEOUT) {
     uv_fatal_error(GetLastError(), "GetQueuedCompletionStatusEx");
   } else if (timeout > 0) {
     uv_update_time(loop);
     if (timeout_time > loop->time) {
       timeout = (DWORD)(timeout_time - loop->time);
       timeout += repeat ? (1 << (repeat - 1)) : 0;
       continue;
     }
   }
   break;
 }
}
https://gist.github.com/77933393b16cdd086c201dd16e77184f

`epfd`: This holds the file descriptor to which the events from it is to be polled.

`events`: This an array of epoll_event structures, it holds the events that are in the ready state.

`maxevents`: The maximum number of events to poll. It must be greater than 0.

`timeout`: The amount of time to poll for completed events.

if `timeout` is `-1`, `epoll_wait` will block indefinitely. But when the any of the events for the file descriptor `epfd` becomes ready it will break off and return.

if `timeout` is `0`, `epoll_wait` will check for any ready event and return immediately.


The return value of epoll_wait is an integer.

if an error occured it returns `-1`.
if the time expires and there are no ready events, it returns `0`.
if the time expires and there are ready events in the file descriptor `epfd`, it returns the number of the completed events in `epfd`.

Now, we have seen `epoll_wait` in its entirety. Let's look at the `uv__io_poll`.

https://gist.github.com/6912691a80ecd12e061b495980687f66

It loops through a `watcher_queue`, `watcher_queue` contains a list of all file descriptors that have an event that needs to be listened for.

It adds the file descriptor to the `backend_fd` using an `epoll_ctl` function.

Next, calls `epoll_wait` with the supplied timeout. The `nfds` integer variable holds the result of the operation. It loops through the completed events and calls their callbacks.

_BSDs, MacOSX_

These OSes use the kqueue function. It has the exact implementation as __Linux__ systems

https://gist.github.com/c91636a82741ec34220ce4bd45625a4a

The difference is where Linux uses `epoll_wait`, `kevent` is in place there.

On initialization, the `uv__kqueue_init` function creates a new kernel event queue and returns a descriptor `loop->backend_fd` by calling the kqueue function.

https://gist.github.com/eee5382d7526fab48605cb40c1ae2b83

Now with this event queue, the kevent function is called to register events with the queue, and return any pending events to the user.

So, in the `uv__io_poll` libuv calls kevent

https://gist.github.com/892e8a7d9106fc1835968c2129342719

to return the number of completed events registered on the file descriptor `backend_fd`'s event queue and run their callbacks.

Let's look at their signatures:

https://gist.github.com/666102b05a34df300105f1152a8893c9

`kqueue` takes no parameters. Its return value is an integer, a positive integer if it successfully created the keveent queue, or `-1` if the creation failed.

`kevent` registers events with the new queue and returns pending events.

`kq`: The number of the file descriptor's event queue.

`changelist`: pointer to an array of kevent structures.

`nchanges`: the size of changelist.

`eventlist`: pointer to an array of kevent structures.

`nevents`: determines the size of eventlist.

`timeout`: The time kevent will poll for pending events on kq.


Now, we have seen different implementations of `uv_io_poll` for different OSes. The `uv_io_poll` takes in a second parameter `timeout` after the `uv_loop_t` structure. This is the time each OS poll functions should block for IO. If the time is zero, the poll function returns immediately. How this timeout is calculated is an intresting part.

The timeout is calculated by `uv_backend_timeout`.

https://gist.github.com/f91510288e55781e6a853571f277505e

* if the `stop_flag` is set that equals `1` therefore, poll exits.
* if there is no active handles and no active requests, then, poll shouldn't wait for anything, it just exits.
* if the idle queue is not empty, IO poll should not wait.
* if pending requests queue is not empty, poll should not wait.
* if there are closing handles, poll should not wait.

if none of the conditions are met, then `uv__next_timeout` is called to determine the timeout value. Onto its implementation:

https://gist.github.com/00ea4de4118f78bd64ee33d77e823af4

We learned earlier that timers are sorted by the nearest timer to expiration.
So here, it gets the minimum timer from the timer_heap, if the heap is empty it returns -1, that is `infinity`.

If the timer has expired, it returns `0`.

If the timer is not yet due, that's still in the future, it gets the difference of the timer and the loop time (the current time) and returns it. If the timer is due 500ms and the loop time is 100ms, therefore IO should poll for 400ms because after the polling the timer will be due by then.


## Microtask queue and nextTicks queue

We said in the `Event Queue` section that we will come to this. Finally, we are here.

These queues are run immediately afer each phase in the event loop.

https://gist.github.com/5a9c9ded11300df3236866ef3676a811

nextTicks have a high priority over microtasks. This means that callbacks scheduled by nextTicks are run before the callbacks set by microtasks are run.

nextTicks are scheduled by `process.nextTicks(()=>{...})`, etc.
microtask by Promises, etc.

# Common Misconceptions

There are many misconceptions about Node.js and Event Loop.

Few of the misconceptions are that:

1. Event loop is inside JS Engine
2. There is a single stack or queue
3. Event loop runs in a separate thread
4. Some async OS api involved in setTimeout

`1.` The event loop is not inside the JS engine. The event loop is an entity that runs when the provided script has been compiled and run by the `v8` compiler.

`2.` The concept is more of a linked list, not a stack or queue.

`3.` Event Loop runs in the main thread. There is no separate thread created for the event loop.

https://gist.github.com/22929251dda8f4585d3681bd99f3a18c

Nodejs compiles and runs the script `1.`. Then, it processes the Event Loop `2`. It iterates through the Event loop till it is drained. There is no thread created for the event loop cycles. The script compilation by v8 must exit before the event loop executes.

`4.` There is no OS API for setTimeout. Everything is implemented in Node.js.

# Conclusion

I described the Node.js event loop system on a scale hitherto undreamt of. I also dove into Node.js and libuv sources for better clarity and explanation of the concept.

I think with these you can confidently use Node.js because you know how it works its nuts and bolts.

In the next post in this series, we will look at the __best practices__ you need in your arsenal as a Node.js developer.

Feel free to ask me anything, if you have any questions concerning this post or if there is aanything to be corrected.

Is somebody still here? Hello? anyone still reading? Hello? Hello?

# Appreciation

Much thanks to:

* Nodejs doc [The Node.js Event Loop, Timers, and process.nextTick()](), where I first read about event loop.
* Saul Ibarra Corretge - For his awesome talk [NodeConf EU -  A deep dive into libuv - Saul Ibarra Coretge]() on libuv.

# Social Media

You can follow/contact me through these following links:

* [Email](kurtwanger40@gmail.com)
* [Twitter](https://twitter.com/ngArchangel)
* [Facebook](https://facebook.com/philip.david.5011)
* [Medium](https://medium.com/@kurtwanger40)
* [LinkedIn](https://linkedin.com/en/chidume-nnamdi)