In this article, we will discuss how node-xyz can be integrated to work with RisingStack Trace.
node-xyz is microservice microframework for node. It is a low level , minimal, yet comprehensive toolkit that can be used to easily develop and deploy microservices. node-xyz consists of two main components:
xyz-core
: which is the microframework that we will be using to write our services.xyz-cli
which is the command line tool that we will be using to deploy our services.xyz-cli
can be configured to work with Trace very easily.
This tutorial is using xyz-core v.0.4
and xyz-cli v0.4.1
. Since node-xyz is being heavily developed at the time, there is the possibility of minor changes in the future.
In this tutorial, we will not focus heavily on the details of microservices since it can get pretty complicated and dependent on your application domain. Instead, we will create mock tasks with different types (CPU and IO tasks) that represent real tasks and deploy them as our microservices.
We will create 2 service types:
- a Front service: This node will receive HTTP requests, translates them to internal messages and dispatches the messages.
- a Worker service: which exposes the two tasks explained above.
- a Client service: this node will represent internal clients that might need to use
Worker
's services.
The services will be deployed to port 4000, 6000 and 5000 respectively.
We will also launch a series of attacks to our Front node to represent external clients.
I will explain some of the details of xyz-core
in this tutorial, but I highly recommend reading xyz's Getting Started document to get more familiar before going any further.
Let's start writing the microservices:
https://gist.github.com/ded9d4598ac139a8c002c08cbd463bbb
Even for this small example, it's worth mentioning that placing an IO intensive and a CPU intensive thask in one process is not a good idea. Nonetheless, the aim of this tutorial isn't to teach you about microservices. It's about xyz and Trace
This node is a bit simpler since we will not .register
any service. Instead, we will use .call()
to send messages to Worker
's tasks.
https://gist.github.com/a9047b6ee0df5412ff1b3b2559922469
Note that:
- For simplicity, all messages are empty and don't have any payload.
- No need to say, the path of ther service, like
/task/io
, is its identifier and should be used by a caller to reach to correct callee. - As you might've seen, none of the nodes are informed about the
IP:PORT
of one another. They use a seed node to join the system. This procedure is a part of xyz's Ping mechanism which helps services discover and explore each other.
At this point, we can have a small test. We have created two services, namely client.ms@127.0.0.1:5000
and worker.ms@127.0.0.1:6000
and we can run them both to see them working. For now, we will not use xyz-cli
and run them with node
command.
Start with:
https://gist.github.com/4ea77607133e60b86b60a0cca7d512c2
You will see iterative logs that indicate task/cpu
and task/io
are not found. This is fine since we haven't launched worker.ms
yet!
Next, in a new terminal:
https://gist.github.com/0086c096e0ab394d5b92c11f57c46054
After a few seconds you will see:
https://gist.github.com/ea912280bb427961d7300b18f0e158ce
Which indicates that the two services are now synced. Rest of the logs should show things like:
https://gist.github.com/5851b0d52a2b42365a06c04dbc931342
Let's write the last service, Front
A front service should usually accept requests from external clients and translate them into internal messages. This process can be quite complicated depending on the business logic of the entire system. Yet again, we will use just a simple form of translation for simplicity. In the front service, an Express server will accept only post requests with /service?service_path=SERVICE_PATH
url. It will also send responses to the external clients since all messages are HTTP and we simply can do that (note that this wouldn't have been easily possible with UDP or Async messaging). Let's see the code:
https://gist.github.com/68977ee67acb9405a1fba9b685f05c2e
If you run $ node front.js
, and send a request to it like:
https://gist.github.com/e4c4a396ea2997f61e389d1bd88337f3
You will get
https://gist.github.com/77193419a300053b596d20d60b089d54
Which is totally fine since front
hasn't connected to other nodes using its seed node. If fact, you can see how front
has responded locally to the message:
https://gist.github.com/fc4f4ca1e7cff6b4aa3990e950f74a78
Which is logged after Front fails to execute
https://gist.github.com/86e2905deb388b0156b4059515966eae
Notes about all three kinds of services:
- Internal messages are sent via
Client
instance while external message are gone throughFront
- Note that all nodes use their
selfConf.host
as their global identifier, so be sure to replace that with a valid static and remote IP if you are testing this code in a VPS. - All nodes will have a
selfConf.seed
instead of havingsystemConf.nodes
to sync and discover other services. This is because we want to deploy more instances of each service later. - While Sync message passing is fine for external clients, usually it is preferable to use an Async messaging for internal messages. In this tutorial, we will use HTTP Sync message for all communications for simplicity. xyz provides some Async messaging mechanisms that you can use.
You can now run the three nodes individually and test them. They should work just fine. But we want to do it easier, that is why we will use xyz-cli
. You can install this module globally using:
https://gist.github.com/3a8f83e2334d593850572e16e2f3bea5
One of the commands in xyz-cli is dev
. This command will run a bunch of microservices according to a config json file. This config file is pretty simple and self-explanatory:
Create a file named xyz.json
:
https://gist.github.com/d3f7145d3f67bfa4b1988ee2ec83801e
Notes:
- CLI will override ports of you provide them. In this case, we are giving them with the same values that they had in
selfConf.transport.0.port
. You can read this page to learn more about how xyz overrides configurations such as ports. - each object in
nodes[]
accepts more options, but we are not filling them now for simplicity. stdio
indicates the destination of each process'sstdout
andstderr
. By default, it'sconsole
which is not good now because the logs of all nodes will be printed in one terminal. If you usefile
value for this key, like now, a new folder namedlog
will be created and each process' log will be written to a separate file. This is much more reasonable.
with this file in the root of all of your files (since our services are simple now we use a flat file structure):
https://gist.github.com/547dfe630953c8701d0823c8039b98e3
you can run:
https://gist.github.com/eb22fee818366697e8ad619bce5a923c
After this, a list of all nodes should be printed:
https://gist.github.com/987299f7ffbe03798722b835de5f71d0
xyz-cli is an interactive command line tool, meaning that you can keep entering commands to get more info and do more stuff, like killing a node, creating a new one and... You can further investigate your nodes with commands such as top:
the message rates given make sense because the front
node is basically idle at the moment, and the client
is sending messages to worker
(~13msg/sec).
Wondering what
Ping Interval
is? See this page.
You might notice that all nodes, regardless of their
stdio
status, will start writing their outputs to file/terminal only after they have been initialized. This is a clean approach, yet it kinda makes it hard to catch exceptions at runtime, like things as simple as a typo. You can always add a-e
toxyz dev
command if you sense that something is wrong. This flag will cause all nodes to output their logs to the current terminal during initialization phase, which is very critical. So if any of them have an exception, you'll see it.
We can talk for another hour about xyz-core and xyz-cli, but as you might recall from the title of this article, this wasn't our main goal!. Let's keep things simple like this and switch this project to be monitored by Trace.
PS. In case you actually want to read for another twenty minutes about xyz you can read these two articles on Medium.
While it's just about... ok to see a list of nodes deployed with xyz-cli, we can't ignore that it's a basic tool. If you want to deploy multiple nodes across multiple servers, xyz-cli will lack some functionalities. To fulfill this requirement, Trance is a perfect solution and can be integrated with xyz-cli with no hassle.
In order to get started with trace, you can sign up for a 14 day trial. This will be just enough to get us started with this service. Next, you should read the Getting Started section. As mentioned, there are two ways to integrate a node process with Trace:
- using environment variables
- using a config file.
While both are possible, using the former solution is much easier. So we will choose that.
After signing up with Trace and creating an infrastructure (which I've named node-xyz-test
), you should create services. For this tutorial, we will create 3 services, concomitant with the three services that we have:
worker
client
front
Per each service, you will receive a set of instructions to follow in order to integrate the process with Trace. Each service will get a unique name
and an apiKey
. The apiKey is common among all services in an infrastructure.
Next, we must include Trace module in all of our services. You must only add the following line at the beginning of each file, give that you have installed @risingstack/trace
:
https://gist.github.com/e501ebc466e5e4626be71087f3c5a25d
Finally, you should add the service name and apiKey to each process at runtime. Hopefully, xyz-cli
provides a handy way to do this. You can just add one key, named env
, to each node in xyz.js
:
https://gist.github.com/3710f784cc7d0e2ccd0e133b3d16562b
You must only replace {YOUR_API_KEY}
inside the file.
Reminder: If you want do deploy nodes across multiple servers, you must replace
selfConf.host
in each node with the remote address values
And there it is! Out sweet microservices appear inside Trace:
Services should appear in your panel:
The Topology should also start to shape:
With message rates being added within a second:
Also, don't forget to see how the Metrics page to see some important information.
Notes:
- During this tutorial, I deployed all services in a single VPS and ran a benchmark test on them using Apache Benchmark, aka. ab. That's why you see an external client sending messages to
Front
. The benchmark was similar to this:
ab -k -p post.data -c 10 -n 20000 HTTP://SERVER_IP:4001/service\?service_path\=task/cpu
post.data
file is irrelevant since we are not using the post body but the query instead.
Remember that Front
node was listening for external clients on port 4001.
- Aside from the thick lines between
- Client -> Worker
- External -> Front | Front -> Worker
which were our job, what are the other thinner lines indicating small message rates (~10rmp)? what about the circulating lines from one node to itself? Those are because of the Default Ping mechanism in xyz. This mechanism basically keeps track of all nodes inside a system and what functions they are exposing, hence it need to send some messages under the hood every once in a while to check other nodes. This is why you can simply call ms.call({service: ...})
and the message will be redirected automatically, even if the destination is in another host all the way across the globe.
- During the peak of response time in the last image, I was taking heavy benchmarks from
?service_path\=task/io
, which is more intensive.
You can read the full documentation of xyz and Trace if you want to learn more!
The full code of this tutorial can be obtained from here.
I hope that you have enjoyed this article. I will sone publish the second part of this article, in which we will scale out our worker service and use an internal load balancer to distribute the messages among them.
In the meantime, any comments or suggestions are welcomed.