When talking about creating a lot of microservices for one workflow:
Again, this is something that looks good on paper, but immediately leads to several questions:
Do you feel the need to deploy six applications to process 1 xml file?
Are these microservices really independent from each other? They can be deployed independently from each other? With different versions and API schemes?
What does the plausibility-microservice do if the validation microservice is down? Is the system then still running?
Do these microservices now share the same database (they sure need some common data in a database table) or are you going to take the even bigger hammer of giving them all their own database?
And a ton of other infrastructure/operations questions.
While arguments can be about the simplicity of those diagrams, you now definitely have these additional operational challenges to solve:
Don’t just need to deploy one application, but at least six.
Maybe even databases, depending on how far you want to take it.
Have to make sure that every system is online, healthy and working.
Have to make sure that your calls between microservices are actually resilient (see How to make a Java microservice resilient?)
And everything else this setup implies - from local development setups to integration testing
Recommendation:
Unless:
you are Netflix (you are not)…
you have super-strong operation skills: you open up your development IDE, which triggers a chaos monkey that DROPs your production database which easily auto-recovers in 5seconds
or you feel like @monzo in giving 1500 microservices a try, simply because you can.
→ Don’t do it.
In less hyperbole, though.
Modeling Microservices:
Trying to model microservices after domain boundaries is a very sensible approach. But a domain boundary (say user management vs invoicing) does not mean taking a single workflow and splitting it up into its tiniest individual pieces (receive XML, validate XML, forward XML).
Polyglot Programming Ecosystem
What might look like fun from a developer’s perspective (developing a perfect system with your perfect language in an isolated setting) is basically never what an organization wants: Homogenization and standardization
.... Historically standardization went way too far. .... . But going full-on polyglot is pretty much the same thing, just the other side of the same coin.
Recommendation :
If you are going polyglot, try smaller diversity in the same programming language eco-system. Example: Kotlin and Java (JVM-based with 100% compatibility between each other), not Haskell and Java.
How to use Build Tools, SSH & Ansible for Java microservice deployments: On relying on simple and proven build and deployment tools
If you are not fixated on creating a breathing cloud of ever auto-load-balancing servers, chaos monkeys nuking your machines, or the warm and fuzzy-feeling of seeing ZooKeeper’s leader election working, then this setup will take you very far.
Oldschool, boring, but working.
On resilience
Whereas a method call execution is basically guaranteed (with the exception of your JVM exiting abruptly), a network call is, by default, unreliable.
Handling errors, not just the happy-cases, is expected for any program. It is the same for microservices, even though you have to take extra care to keep all of your deployed API versions compatible, as soon as you start with individual microservice deployments and releases. And if you want to go full-on chaos-monkey, you will also have to live with the possibility that your servers just get nuked during request processing and you might want the request to get re-routed to another, working instance.
On choosing a proper (message broker) solution
In general, though, try to dismiss any artificial performance reasons when choosing your broker. There was a time when teams and online communities argued a ton about how fast RabbitMQ was and how slow ActiveMQ was.
Now you are having the same arguments on RabbitMQ being slow with just a consistent 20-30K/messages every.single.second. Kafka is cited with 100K messages/a second. For one, these kinds of comparisons conveniently leave out that you are, in fact, comparing apples and oranges.
But even more so: Both throughput numbers might be on the lower or medium side for Alibaba Group, but you author has never seen projects of this size (millions of messages every minute) in the real world. They definitely exist, but these numbers are nothing to worry about for the other 99% of regular Java business projects.
So, ignore the hype and choose wisely.
your team might decide (and maybe even convince businesspeople): That is way too simple and boring, instead of a login service, let’s write a really capable UserStateChanged microservice - without any real, tangible business requirements. And because Java is currently out of fashion, let’s write the UserStateChanged microservice in Erlang. And let’s try to use red-black trees somewhere, because Steve Yegge wrote you need to know them inside-out to apply for Google.
From an integration, maintenance and overall-project perspective, this is just as bad as writing layers of spaghetti code inside the same monolith.
Fabricated and over-the-top example? Yes.
Unfortunately, also not uncommon in real-life.
depending on your organization, trust and communication levels, this can lead to a lot of shoulder-shrugging and blaming, if a random part of the whole microservice chain breaks down - with no-one accepting full responsibility anymore.
depends heavily on company size, with the general rule: The bigger, the more problematic.
- Who is working on microservice number 47?
- Did they just deploy a new, incompatible microservice version? Where was this documented?
- Who do I need to talk to for a new feature request?
- Who is going to maintain that Erlang microservice after Max left the company?
- All our microservice teams work not only in different programming languages, but also in different time zones! How do we coordinate properly?
Going full-on Java microservices is one end of a pendulum. The other end would be something like hundreds of good old Maven modules in a Monolith. You’ll have to strike the right balance.
I'll keep saying this ... if people can't build monoliths properly, microservices won't help. #qconlondon #DesignThinking #Modularity
- Simon Brown
I can’t explain how horrible it feels when the team spends 70% of the time fighting with this modern infrastructure setup and 30% of the time on actual business logic.
- Siva Prasad Reddy