In this day and age, it's quite common to provide real-time updates to users. What app doesn't use spinners, blinking notifications, or progress bars to indicate something is happening?
In ancient times, we used polling to solve this. We nagged the server periodically to check if something had changed, but these days it's much more common to use WebSockets (WS). The downside is that having an open socket requires a persistent connection. This rules out most serverless compute solutions on the market as these generally disappear into the ether after doing their thing.
Fly is kinda like serverless containers running at the edge. Applications are deployed to multiple regions and Fly will spin containers up and down depending on load. Because of this, I initially thought it wouldn't be possible to use WebSockets on Fly.
In this article we're going to see how to architecture a global notifications system on Fly using WebSockets and Google Cloud Pub/Sub. Each user will be able to alert all other connected users in real-time of alien landings on Earth. You wouldn't want to miss an alien landing, right?
Fly applications can be configured to run in one or multiple regions. Requests will be routed to the closest available region via Fly's routing layer.
The total number of virtual machines (VMs) that Fly will run for a given application will depend on the app's scaling configuration. By default, Fly apps will run a single VM regardless of the configured regions. The number of VMs can be increased manually and will be distributed evenly across regions.
If autoscaling is enabled, Fly will manage the configured total number of VMs depending on traffic. As the number of concurrent requests on a region grows, Fly will spin up more VMs in that region to be able to satisfy the demand. In consequence, when autoscaling is enabled, your requests will rarely hit the same the same VM every time on a hot region.
Regardless of your scaling settings, this process is totally seamless. From the outside world, the application is just a domain or IP address answering requests.
Fly's networking layer handles all the data packets that go in and out of an application, so it will know whenever a client connects and disconnects from a VM. As long as there's one WS client connected to a VM, Fly will keep it alive.
If the WS connection is interrupted, clients will probably try to reconnect and may land on the same VM, or not. It will all depend on the scaling settings and number of concurrent connections in a particular region.
Client-side, this process is completely seamless. As mentioned before, from the outside world, a Fly app is just a domain.
Server-side, you won't need anything special either. It will all work as expected as long as your Fly application is stateless. This is not specific to using WS as it's assumed a Fly VM might shut down at any time. Either because of the scaling settings, or because a new version of the application is deployed.
Now that we've seen an overview of how it all works let's see a more practical example.
If we just wanted a single WS client and a single Fy VM to talk to each other we could simply use a typical WS setup.
As explaine before, connecting a client to a Fly server using WS works pretty much as you'd expect:
const server = 'ws://my-fly-app.fly.dev/ws'
const socket = new WebSocket(server);This is not enough if we want to be able to communicate between clients as these will most likely be connected to different VMs.
To solve this, we're going to use Google Cloud Pub/Sub, a messaging service that will allow us to broadcast messages from our WS clients to all our Fly VMs, and in turn to their connected clients.
As it name implies, Pub/Sub follows the publisher/subscriber pattern. Each Pub/Sub project has a number of topics. Topics have subscriptions, and each subscription has one or more subscribers.
Messages can be published to topics, and Pub/Sub will ensure they will be received by at least one subscriber of every subscription.
For our use case, we want to ensure all our Pub/Sub messages are sent to all our Fly VMs, so we need one subscription (and subscriber) for every VM.
Fly VMs receive a number of environment variables when booting up. One of those is FLY_ALLOC_ID, a unique UUID identifying a VM. This is perfect, as it will allow us to dynamically create subscriptions such as:
projects/<googgle-cloud-project-name>/subscriptions/ID_<VM-UUID>
Note that we need to add ID_ in front of subscription name to ensure it always with a letter and comply with the Pub/Sub resource name rules.
Once our VM shuts down, we can simply delete the subscription. It's possible something could go wrong (eg: alien attack) and a VM could crash or shut down without deleting the subscription. Not to worry, Pub/Sub will not send any messages to a subscription without subscribers and it will be deleted after 31 days of inactivity.
The whole thing looks like this:
- A WS client sends a notification to a Fly VM with an alert that there's an alien landing in the vecinity
- The WS server will forward the message to the
ALIEN_NOTIFICATIONStopic - Pub/Sub will send a copy of the message to each subscription
- Since there's one subscription per VM, each VM will receive a Pub/Sub message and forward it to every WS client
To deploy the demo app to Fly you're going to need:
- A Google Cloud account
- A Fly account
- The latest version of the Fly CLI installed
To setup Pub/Sub, first go to the Google Cloud console and create a new project.
I'll name my project fly-alien-landings:
Once that's done, activate the Pub/Sub service for this project by simply navigating to the service from the console main menu.
Once activated, we don't need to do anything else. Our application will automatically create the needed topic and subscriptions at runtime.
We now need to create a service account to allow our Fly application to interact with our Google Cloud project. From the main menu, open the IAM & Admin section, and go to Service accounts:
Then, create a new service account, and give it the role of Pub/Sub Admin:
Finally, open the new service account, go to the Keys section, and create a new key of JSON type:
This will prompt a download for a .json file. Save it somewhere safe as we will need it later on.
Do not commit this file to your Git repository.
Now that we're done with Pub/Sub, let's focus on our application.
We can now either clone the repository with the application by doing:
git clone https://github.com/PierBover/fly-web-socket-google-cloud-pub-sub.git
Or download a .zip with the project from Github.
Once the repo is in your dev machine, copy the .json key file into the project folder and rename it to google-service-account.json.
Before we can use the Fly CLI we need to authenticate with Fly by doing:
fly auth login
Next, open the project folder with a terminal and simply do:
fly launch
The CLI is smart enough to understand this is a Node project. It will create a default fly.toml configuration file using the Node Buildpack (similar to Heroku Buildpacks).
You will be asked to give the app a name or use a provided name. Your app will be available at:
https://<app-name>.fly.dev/
The CLI will also ask you if you want to deploy the app. You can do it now, or later with:
fly deploy
Initially, our Fly application will be configured to run a single VM in a single region. We can verify that by listing the the app's regions with:
fly regions list
We're now going to add a couple more regions so that our notification system is truly global. The command to do this is:
fly regions add <region-code>
Check the docs with the full list of Fly regions and their codes.
With this in mind let's add 4 more regions.
First let's add Amsterdam:
fly regions add ams
Then Singapore:
fly regions add sin
Then Los Angeles:
fly regions add lax
Let's also add Sydney (they probably have aliens down there too!):
fly regions add syd
If we now list our regions with fly regions list we should have something similar to this:
Region Pool:
ams
dfw
lax
sin
syd
(dfw is my initial region but yours will probably be different)
As I mentioned before, although we have configured multiple regions, Fly is only allowing one VM instance to run.
We can verify that by doing fly status:
Instances
ID VERSION REGION DESIRED STATUS HEALTH CHECKS RESTARTS CREATED
67a0fec8 7 syd run running 1 total, 1 passing 0 6m50s ago
As you can see, the Fly is only running a single VM on the Sydney region.
Let's now tell Fly we want our application to run five VMs :
fly scale count 5
This will return:
Instances
ID VERSION REGION DESIRED STATUS HEALTH CHECKS RESTARTS CREATED
5bc3e3ad 8 ams run pending 0 5s ago
3f2df1d3 8 dfw run pending 0 5s ago
d6bbc9a8 8 lax run pending 0 5s ago
5c42daf4 8 sin(B) run pending 0 5s ago
67a0fec8 8 syd run running 1 total, 1 passing 0 7m11s ago
Because we've allowed more VMs to be activated, Fly will now distribute those across all configured regions.
If we do fly status again a minute later, we'll see our application is now running in all regions!
Instances
ID VERSION REGION DESIRED STATUS HEALTH CHECKS RESTARTS CREATED
d6bbc9a8 8 lax run running 1 total, 1 passing 0 10m29s ago
5c42daf4 8 sin(B) run running 1 total, 1 passing 0 10m29s ago
5bc3e3ad 8 ams run running 1 total, 1 passing 0 10m29s ago
3f2df1d3 8 dfw run running 1 total, 1 passing 0 10m29s ago
67a0fec8 8 syd run running 1 total, 1 passing 0 17m35s ago
To access the app, simply open the the browser on:
https://<app-name>.fly.dev/
You should see the UI of the application:
After clicking the red button, the client will send a WS message the VM, which in turn will send a Pub/Sub message to the ALIEN_NOTIFICATIONS topic.
The Pub/Sub message will be received by all active VMs (including the one that sent it) and then re-sent to all their connected WS clients.
So there you have it, a global notification system that will scale automatically to be able to handle any load. The more regions you add, the lower the latency between the servers and the clients will be.
Now we only have to wait for an alien landing!