About 6 years ago when I was a PHP ecommerce dev, I've always wanted to work with distributed systems and microservices. They seemed so cool and the new way of architecting software systems because all of big tech did it. But now working with them day to day for about 3 years I came to realize that my initial anticipation was stupid. Well, not stupid in the traditional sense, because how could I have known what I'd get myself into.
Microservices and distributed systems have been preached to solve two major problems: Scale and scale.
So let's talk about them.
The first thing you've probably heared about microservices is, that they scale very well. Since you have many of them you can precisely dial the knob, that deploys more of the thing, that handles some piece of your system. Or in more concrete terms. If you have a bunch more people trying to order something, you can scale up the order service.
What's the problem with this?
Microservices usually don't live in isolation. There's always another service talking to it and it will talk to others. Whether that's through an HTTP, RPC, or a message bus. You might as well call messages "events" or "commands" but that doesn't really matter. It always invokes some kind of other process.
Now if you have a ton of things to process (e.g. orders), you'll usually issue a ton of HTTP requests. Each of them taking a few milliseconds to be delivered and processed/acknowledged. Sometimes even 10s or 100s of milliseconds, depending of how many layers of networking and virtualizations you have to go through.
This is a lot of overhead.
Not once have I seen an HTTP service accept requests in bulk, meaning that I could send it 1000 orders to process at once. This is largely because, we tend not to think in bigger quantities, rather individual resources. Or how Casey Muratori frames it: "Where there's one, there's many".
Had you not separated out the order processing into a microservice, this thing would just be a function call, inside your code. And to nobody's surprise, function calls are much much faster than HTTP calls, like instantaneous. I'll leave it up to you to calculate how much time it takes for a CPU with a 3GHz clock speed to execute 3-5 assembly instructions.
Apart from that, if you didn't have a microservices, you could as well scale up the entire system as a whole, depyloing multiple instances of. So the only real benefit in terms of scaling you can get from microservices is, that you can "underscale" certain parts of your system. But when seeing another outrageous AWS bill posted on HN, I get the feel that nobody really cares about this fact.
I'm sure that you've been aware of this all along and might be thinking: "Microservices don't exist for this kind of scale." So let's talk about that.
Once you're past the point of thinking microservices increase performance, you come to the conclusion that they help an organisation scale. When you have hundreds or thousands of engineers you want to find a way for them not to step on each others toes. And by that I mean writing and integrating new code into an existing codebase.
How often have I created a PR, patiently waiting for all the CI runners to give me green light, only to find out at the very end, that my branch is now out of date. Then having to rebase and hoping my turn will come this time.
This is very frustrating, which is why we split pieces of our system into separate components and then creating microservices out of them. You could think that this a big achievement, because now I don't have to care about what people are doing in auth service for me to change my order service.
However, if we take a step back and think about how any of this has worked before we had microservices, you might realize that microservices are not the only way to solve this problem.
Whenever we build software, we rarely write everything from scratch. Or can you think of a time in which you implemented the postgres wire protocol, or a JSON parser, or an HTTP request router. Probably not, because we just pulled in a library off of GitHub or NPM and off we went. And whether that library is for parsing JSON or inserting an order into a database doesn't really matter.
But have you ever given it a second thought, that there are real people, hundreds and thousands of them, that wrote this piece of code you're using? How often did you find yourself stepping onto their toes, or vice versa? Never.
They work on their codebase, at their own leisure and you do too. And whenever you needed a new feature from them, you just pulled in a new version.
So, do we really need microservices to solve a organisational scaling problems? I don't think.
In the same manner in which we send an HTTP request to the order service to shovel stuff into the database, we could have effectively made a function call to an order library, developed by another team.
This not only solves the organisational scaling problem, but also the performance problem. You can now issue hundreds and thousands of individual "order create" calls with little to no overhead.
For almost every microservice at $COMPANY, we have a Swagger, OpenAPI, Protobuf file that comes with it. This clearly specifies what HTTP endpoints exist and what parameters you can give them. Out of this, we code-generate an HTTP client library that turns the definition file into function calls. With that, we don't have to issue raw HTTP requests to talk to another service.
So in a sense, we're already "librarifying" our microservices, except it performs the function call through HTTP. Could we instead just put the logic of the microservice right into the library? I don't see why not.
This is how we do it, and I'm pretty confident that many other companies do it in the exact same way.
Let's quickly look at both (microservces and libraries) side by side:
| Microservices | Libraries | |
|---|---|---|
| Scaling the performance | Scale independently | Scale the whole system |
| Scaling the organisation | Independent code ownership | Independent code ownership |
| Inter-service communication | Slow HTTP calls | Fast function calls |
| Infrastructure requirements | Service discovery, API gateways, Many deployment pipelines | Virtually none |
| Debuggability | Distributed Tracing | Debugger |
| Independent deployability | Easy | Difficult |
The promise of microservices and the problems they're supposed to solve come at a great cost that is often overlooked. They indeed allow you to scale the organisation, but the obvious alternative that we've been working with for decades is almost never considered.
Still, they seem to be the dominant approach because we largely emulate what other companies are doing. We look at Google, Netflix or Amazon and try to mimic them, with the hope of having an equal amount of success.
I would like to see more engineers thinking critically about what they want to get themselves into and try to reason from first principles. More often than not, the problems we're facing are not fundamental problems in computing, but rather self-inflicted.
We strive for cool new tech and want to apply it to our organisation and I get that. But what butterfly effects that has down the line can be very harmful. Not only do you have to maintain the system, but also customers who are indirectly affected by your decisions (think of reliability, profit margins, etc).
Still, you might find more reasons why microservices and their trade-offs are still worth it, but generally speaking this organisational scaling problem is the defacto solution microservices exist for.
I don't mind, if you don't want to buy into this, that's fine. I'm also not trying to argue for a perfect solution here.
But try to keep in mind that often times there are equivalent or better solutions to a problem that are way less harmful than what we see from big tech.
The opposite of a "distributed system" is not a "monolith". It's an "integrated system".
So try using that next time instead.
Microservices have been a popular trend in the tech industry. However, many companies may not actually need to adopt a microservices architecture. Just because big tech companies use microservices, it doesn't mean that it's the best approach for every organization.
This situation is similar to the hype around frontend frameworks. Just because many companies are using a particular framework, it doesn't mean that it's the right choice for your specific needs.
BTW, nice points view.