Vertical decomposition. Creating cohesive services
One of the biggest misconceptions about services is that a service is an independent deployable unit, i.e., service equals process. With this view, we are defining services according to how components are physically deployed. In our example, since it’s clear that the backend admin runs in its own process/container, we consider it to be a service.
But this definition of a service is wrong. Rather you need to define your services in terms of business capabilities. The deployment aspect of the system doesn’t have to be correlated to how the system has been divided into logical services. For example, a single service might run in different components/processes, and a single component might contain parts of multiple services. Once you start thinking of services in terms of business capabilities rather than deployment units, a whole world of options open.
What are the Admin UI, Admin Backend and Website UI, Website Backend components? They basically act as containers of services. They are maintained by their own teams and their sole purpose is to coordinate between services. These components are business-logic agnostic.
Avoiding Microservice Megadisasters unsure about the approach to search and data duplication
Microservices and Rules Engines – a blast from the past
"search engine, not search service" "allows each microservice to put a component into it, and the search engine will run that set of rules" "what we are talking about here is not the whole microservice, but the search component of that service" "that way, the search engine doesn't need in and of itself access to all of that data directly"
Don't build a distributed monolith
"Don't couple systems with binary dependencies"
Alas this seems to go against the "thinking of services in terms of business capabilities rather than deployment units" principle. If the deployment is intertwined, is seems that there will be binary dependencies.
The entity service antipattern
Five pieces of advice for new technical leads
in general, re-organizing the architecture of a system is usually possible - if and only if - the underlying data model is sane.
What is the convention for addressing assets and entities? Is it consistent and useful for informing both security or data routing?
What is the security policy for any specific entity in your system? How can it be modified? How long does it take to propagate that change? How centralized is the authentication?
If a piece of "data" is found, how complex is it to find the origin of this data?
What is the policy/system for enforcing subsystems have a very narrow capability to mutate information?
More than concentric layers The Software Architecture Chronicles
Managing the Complexity of Microservices Deployments
Designing Microservice Architectures the Right Way slides
To Test A System, You Need A Good Design Shunt pattern Two-level test suites?
For a test environment, you can inject an “In-Memory Data Source.” For production, you can use the “HTTP Server Data Source.”
How Contract Tests Improve the Quality of Your Distributed Systems
SOLID Architecture in Slices not Layers
For too long we've lived under the tyranny of n-tier architectures. Building systems with complicated abstractions, needless indirection and more mocks in our tests than a comedy special. But there is a better way - thinking in terms of architectures of vertical slices instead horizontal layers. Once we embrace slices over layers, we open ourselves to a new, simpler architecture, changing how we build, organize and deploy systems.
Scaling without cross-functional teams
Growing Object-Oriented Software, Guided by Tests Without Mocks Unit testing anti-patterns: Structural Inspection
Test automation without a headache: Five key patterns
What’s your release process like?
Lessons from Building Static Analysis Tools at Google
Writing Documentation When You Aren't a Technical Writer hn. Semantic linefeeds. guidelines.
automated the checks as much as possible with linters [2].
Conforming container antipattern
Break Up With Your Frontend Monolith - Elisabeth Engel
Compositional UIs - the Microservices Last Mile - Jimmy Bogard
Before using JDepend, it is important to understand that "good" design quality metrics are not necessarily indicative of good designs. Likewise, "bad" design quality metrics are not necessarily indicative of bad designs. The design quality metrics produced by JDepend should not be used as yard sticks by which all designs are measured.
Reconstructing thalia.de with self-contained systems
Optimizing for iteration speed
one of the most scary thing in software engineering is “inventory” of code that builds up without going into production. It represents deployment risk, but also risk of building something users don’t want. Not to mention lost user value from not shipping parts of the feature earlier (user value should be thought of as feature value integrated over time, not as the feature value at the end state).
Unit test your Java architecture tweet
If your primary motivation for building microservices is to enforce modular architectures, think twice. Modularity is solved within the JVM (JPMS, OSGi, JBoss Modules; even multimodule builds get you far), don't pay the price of distributed computing + remote calls just for this.
Apache Kafka als Backend für Webanwendungen?
How Events Are Reshaping Modern Systems by Jonas Bonér
Complex Event Flows in Distributed Systems
Designing Events-first Microservices. Journey to Event Driven – Part 1
Microservices in a Post-Kubernetes Era
In the post-Kubernetes era, using libraries to implement operational networking concerns (such as Hystrix circuit breaking) has been completely overtaken by service mesh technology.
the testing renaissance. tweets about testing.
HANDS-ON INTRO TO KUBERNETES & OPENSHIFT.
Tell Don't Ask. more. How Interfaces Are Refactoring Our Code. The art of embugging. GetterEradicator.
AWS Solution Architect Associate exam.
Hybrid Networking Reference Architectures.
Docs as Code – Architekturdokumentation leicht gemacht.
No More Silos: How to Integrate Your Databases with Apache Kafka and CDC.
Streaming Data Clears the Path for Legacy Systems Modernization.
Streaming MySQL tables in real-time to Kafka.
Streaming databases in realtime with MySQL, Debezium, and Kafka. Mención en: ¡Larga vida al legacy!.
no ser el dueño del modelo de datos debería ser algo temporal
parece que el "sistema secundario" es solo de lectura en un principio
el siguiente pase, de alguna manera, tienes que ser capaz de modificar el sistema antiguo... arquitecturasa más complejas con bidireccionalidad. no uses bus de eventos. expon servicios en el sistema nuevo, haz que el sistema legado los accede. no uses bus de eventos (?)
sin eventos obligo al software legado a saber a dónde me he llevado ese trocito que le he quitado.
[para sincronizar] podemos usar eventos, triggers...
GETTING STARTED WITH DDD WHEN SURROUNDED BY LEGACY SYSTEMS - Eric Evans. bubble context. strategy 1 - bubble context. strategic design. bounded context.
Listen to Yourself: A Design Pattern for Event-Driven Microservices.
For example, you cannot guarantee that a commit to Cassandra and a message delivery to Kafka would be done atomically or not done at all.
Let’s take a common use case: Updating a local NoSQL database and also notifying a legacy system of record about the activity.
However, there is still a concrete problem: How do you guarantee atomic execution of both the NoSQL writes and the publishing of the event to the message broker?
Note: Potential duplicate messages are always a possibility with a message broker so you should design your message handling to be idempotent regardless of the solution you choose.
All your events and database writes must be idempotent to avoid duplicate records.
The client isn’t guaranteed to read their own writes immediately.
The transaction log tailing pattern can achieve similar results to those described here. Your transactions will be atomic without resorting to two phase commit. The transaction log tailing pattern has the added benefit of guranteeing your database is committed before returning a response to the client.
Pattern: Transaction log tailing.
each step of a saga must atomically update the database and publish messages/events. It is not viable to use a distributed transaction that spans the database and the message broker.
How to solve two generals issue between event store and persistence layer?.
Event-Driven Data Management for Microservices.
One way to achieve atomicity is for the application to publish events using a multi-step process involving only local transactions. The trick is to have an EVENT table, which functions as a message queue, in the database that stores the state of the business entities.
Domain events: simple and reliable solution.
In an event-driven architecture there is also the problem of atomically updating the database and publishing an event.
[my thoughs] are the options: 1 - "listen to yourself" pattern and 2 - "keeping an internal events table"? Also 3 - "log trailing"?
Paypal talk. slides. Streaming Data Microservices. Oracle Golden Gate.
from the slides: 66: XA transactions: ensure consistency, give up availability 67: Event Sourcing: give up read-your-writes consistency (Is this the "listen to yourself" pattern?) 68: Change Data Capture: read-your-writes + eventual consistency across systems
OLAP engines like Apache Druid, LinkedIn's Pinot
"Use Change Data Capture, rather than XA Transactions or Event Sourcing, for replicating data between data systems where consistency is required, etc., such as financial services... Also use schemas"
logs, not queues!
Unlike queues, consumers don't delete entries; Kafka manages their lifecycles
N Consumers, who start reading where they want
Akka Streams, Kafka Streams - libraries for “data-centric microservices”. Smaller scale, but great flexibility
Kubernetes: Your Next Application Server. video.
How to extract change data events from MySQL to Kafka using Debezium.
one step forward, two steps back.
Introduction to the Kubernetes Operator Framework
Keynote: Maturing Kubernetes Operators - Rob Szumski.
DEATH OF LOGGING, HEXAGONAL ARCHITECTURES, TECHNOLOGY AND ARCHITECTURES.
Introduction to Cloud Storage for Developers.
building a CI / CD bot with Kubernetes
managing env vars in production comes down to doing one of two things: using an environment file that is securely stored and securely retrieved, or retrieving each key from a secure secrets management service like Vault, Keywhiz or Cyberarc. The former is easier, as it requires less infrastructure, but requires greater care. The latter requires more infrastructure but handles things like role based access for each key more easily
Istio Multicluster on OpenShift
Kafka for long-term storage. so. hn. How Pinterest runs Kafka at scale. Streaming Hundreds of Terabytes of Pins from MySQL to S3/Hadoop Continuously. Is Kafta a database?. experimentation with event-based systems. The Magical Rebalance Protocol of Apache Kafka. The Death and Rebirth of the Event-Driven Architecture. ETL Is Dead, Long Live Streams. event-based architectures with Kafka and Atom. Complex Event Flows in Distributed Systems. Restoring Confidence in Microservices: Tracing That's More Than Traces.
This is an important requirement for processes that calculate real-time results but need to periodically recalculate results (say when their processing logic changes).
Something to keep in mind as well is that cluster restarts (especially after unclean shutdowns) might take a very long time, as all logs would need to be checked at broker startup. Apart from that I can't think of large reasons not to do this, though I agree that dumping data to S3/HDFS/similar should be the preferred solution
we use Kafka to transport data to our data warehouse, including critical events like impressions, clicks, close-ups, and repins. We also use Kafka to transport visibility metrics for our internal services
Language-oriented software engineering. tweet.
The future of Kubernetes is Virtual Machines hn
Developing applications on OpenShift in an easier way
Mastering Spring framework 5, Part 2: Spring WebFlux
Day Two Kubernetes: Tools for Operability.
kubernetes guideposts 2019 Simple Multi-tenancy with Django Running on OpenShift
KUBERNETES FAILURE STORIES. hn.
Cloud native Java EE on OpenShift Adam Bien.
making the most of Kubernetes clusters
Scaling a Distributed Stream Processor in a Containerized Environment
rethinking legacy and monolithic systems
"how do I propagate state across asynchronous, reactive execution pipelines?". video. Spring Tips: Testing Reactive Code. RxJava vs Reactor. Reactive Spring: Eine Einführung in die reaktive Programmierung. Point-to-Point Messaging Architecture - The Reactive Endgame. building reactive pipelines tweet. reactive DDD. How (not) to use Reactive Streams in Java 9+. Assembly time Subscription time Execution time) 404. construyendo pipelines reactivos slides. Spring Tips: Reactive MySQL with Jasync SQL and R2DBC. reactive streams operators. RxJava by example. reactive jdbc tweet. 5 reasons to use RxJava in your projects. reactive programming - lessons learned. reactive transactions. reactive-revolution course materials. marble diagrams. reactive streams and Kotlin flows. Event Driven with Spring. How to build Reactive Server in 50 minutes. moving from imperative to reactive. reactive programming - lessons learned. more. slides. Five Things About RxJS and Reactive Programming. Going full reactive with Spring Webflux and the new CosmosDB API v3. reactive streams basic concepts. Streaming data as one additional use case for #reactive programming. building reactive pipelines. the value of reactive systems. reactor. Do's and Don'ts: Avoiding First-Time Reactive Programmer Mines.
Learn Openshift operator framework
In big companies, 95% of apps are still old school: firewall -- load balancer -- 5 front ends -- 3 back ends -- two database servers.
"everybody wants to get rid of ELK for logging quite soon".
airhacks tv 59 docker vs. openshift effective web standards
Service Catalog and Kubernetes
Kubernetes declarative object configuration model is one of the most interesting features of the orchestrator
12 ways to get smarter about Kubernetes.
Microservices in a Post-Kubernetes Era.
How Kubernetes can break: networking
Automating stateful applications with Kubernetes operators Reaching for the Stars with Ansible Operator
An Incremental Architecture Approach to Building Systems.
Various links about persistence and DDD:
https://tech.transferwise.com/hibernate-and-domain-model-design/ https://stackoverflow.com/questions/10099636/are-persistence-annotations-in-domain-objects-a-bad-practice https://stackoverflow.com/questions/14737652/entity-objects-vs-value-objects-hibernate-and-spring https://stackoverflow.com/questions/31400432/ddd-domain-entities-vo-and-jpa https://stackoverflow.com/questions/2597219/is-it-a-good-idea-to-migrate-business-logic-code-into-our-domain-model https://stackoverflow.com/questions/821276/why-should-i-isolate-my-domain-entities-from-my-presentation-layer https://softwareengineering.stackexchange.com/questions/350067/is-it-good-practice-to-use-entity-objects-as-data-transfer-objects https://softwareengineering.stackexchange.com/questions/378866/understanding-ddd-when-using-an-orm-such-as-hibernate https://softwareengineering.stackexchange.com/questions/171457/what-is-the-point-of-using-dto-data-transfer-objects https://softwareengineering.stackexchange.com/questions/140826/do-orms-enable-the-creation-of-rich-domain-models https://blog.pragmatists.com/refactoring-from-anemic-model-to-ddd-880d3dd3d45f https://enterprisecraftsmanship.com/2016/04/05/having-the-domain-model-separate-from-the-persistence-model/
Custom Implementations for Spring Data Repositories.
Three-Part Architecture of the Next Generation Data Center Inside NetApp.
Conquering the Challenges of Data Preparation for Predictive Maintenance
Java 9: Bessere Domänenmodelle mit Java-9-Modulen.
Links about the strangler pattern https://news.ycombinator.com/item?id=19122973 strangler pattern https://news.ycombinator.com/item?id=19125333 https://paulhammant.com/2013/07/14/legacy-application-strangulation-case-studies/ https://www.michielrook.nl/2016/11/strangler-pattern-practice/ https://trunkbaseddevelopment.com/strangulation/ https://www.leadingagile.com/2018/10/the-urge-to-stranglethe-strangler-pattern/ https://www.martinfowler.com/bliki/StranglerApplication.html https://twitter.com/martinfowler/status/357142664665251841 https://blog.overops.com/strangler-pattern-how-to-keep-sane-with-legacy-monolith-applications/ https://blogs.sap.com/2017/09/25/strangler-applications-monolith-to-microservices/
Links about DTO mappers https://auth0.com/blog/automatically-mapping-dto-to-entity-on-spring-boot-apis/ https://www.baeldung.com/entity-to-and-from-dto-for-a-java-spring-application http://modelmapper.org/ https://medium.com/@hackmajoris/a-generic-dtos-mapping-in-java-11d649b8a486 https://stackoverflow.com/questions/2828403/dto-and-mapper-generation-from-domain-objects https://stackoverflow.com/questions/14523601/bo-dto-mapper-in-java https://stackoverflow.com/questions/15117403/dto-pattern-best-way-to-copy-properties-between-two-objects https://stackoverflow.com/questions/1432764/any-tool-for-java-object-to-object-mapping https://stackoverflow.com/questions/678217/best-practices-for-mapping-dto-to-domain-object https://codereview.stackexchange.com/questions/64731/mapping-interface-between-pojos-and-dtos https://softwareengineering.stackexchange.com/questions/171457/what-is-the-point-of-using-dto-data-transfer-objects https://www.jhipster.tech/using-dtos/ http://appsdeveloperblog.com/java-objects-mapping-with-modelmapper/ http://www.adam-bien.com/roller/abien/entry/creating_dtos_without_mapping_with The Ping class is a JPA entity and JSON-B DTO at the same time: http://www.adam-bien.com/roller/abien/entry/creating_dtos_without_mapping_with DTOs are also motivated by their typesafe nature. Lacking typesafety, JSON-P JsonObjects are not used as DTOs. https://www.credera.com/blog/technology-solutions/mapping-domain-data-transfer-objects-in-spring-boot-with-mapstruct/ https://rmannibucau.wordpress.com/2014/04/07/dto-to-domain-converter-with-java-8-and-cdi/ https://vladmihalcea.com/the-best-way-to-map-a-projection-query-to-a-dto-with-jpa-and-hibernate/ https://github.com/porscheinformatik/anti-mapper jpa hashCode euquals dilemma
Running your own DBaaS based on your preferred DBs, Kubernetes operators and containerized storage.
Microservices in a Post-Kubernetes Era
In the post-#Kubernetes era, using libraries to implement operational networking concerns (such as Hystrix circuit breaking) has been completely overtaken by service mesh technology.
Netflix Titus, Its Feisty Team, and Daemons.
Scaling a Distributed Stream Processor in a Containerized Environment
Odo.
Is Shared Database in Microservices actually anti-pattern?. hn
The Whys and Hows of Database Streaming
The Changing Face of ETL: Event-Driven Architectures for Data Engineers
Your migrations are bad, and you should feel bad. hn.
An introduction to distributed systems
Paying Technical Debt at Scale - Migrations @Stripe.
Transaction scripts https://dzone.com/articles/transaction-script-pattern https://stackoverflow.com/questions/16139941/transaction-script-is-antipattern https://gunnarpeipman.com/architecture-design-patterns/transaction-script-pattern/ https://learnbycode.wordpress.com/2015/04/12/the-business-logic-layer-transaction-script-pattern/ http://www.servicedesignpatterns.com/webserviceimplementationstyles/transactionscript http://lorenzo-dee.blogspot.com/2014/06/quantifying-domain-model-vs-transaction-script.html http://grahamberrisford.com/AM%202%20Methods%20support/06DesignPatternPairs/Domain%20Driven%20Design%20v.%20Transaction%20script.htm
Automating applications with @kubernetesio operators
Code in the database vs. code in the application. 2. 3. 4. 5. 6. 7 by Lukas Eder. tweet. 8. reddit. 9. 10. 11. mf.
The big myth perpetrated by architects who don’t really understand relational database architecture (me included early in my career) is that the more tables there are, the more complex the design will be.
Kafka running on OpenShift4 using Ceph Block Storage
12 Factors for Cloud Native and Openshift.
data preparation for predictive machine learning
spring high performance batch processing
Installing Openshift 4 from start to finish. Multiple stages within a Kubernetes cluster.
Migrating a Retail Monolith to Microservices: Sebastian Gauder at MicroXchg Berlin. slides.
Idempotency - challenges and solutions over HTTP
reflections on moving to Kubernetes. advanced kubernetes. when to use kubernetes.
Bringing up an OpenShift playground in AWS
Should that be a microservice? hn.
deploy != release. Testing in Production, the safe way. Deploy != Release (Part 1). Deploy != Release (Part 2). Istio Observability with Go, gRPC, and Protocol Buffers-based Microservices. works in staging. Using Blue-Green Deployment to Reduce Downtime and Risk . NoStaging. How to Deploy Software with Envoy. Reactive REST API Using Spring Boot and RxJava.
Mature Microservices and How to Operate Them.
Reconciling Kubernetes and PCI DSS for a Modern and Compliant Payment System.
become a better software architect
Eoin Woods on Democratising Software Architecture at ICSA 2019
Software architecture is still needed because stakeholders are still around, we need to decide on design tradeoffs and we have several cross cutting concerns in software. In practice, what happens nowadays is having more empowered cross-functional teams and using more lightweight descriptions for architecture than in the past. Difficult to understand and evolve architecture diagrams are now replaced by lightweight C4 and Architecture Decision Records diagrams. Code static and runtime analyses combined with informal documentation in the form of Wiki or powerpoint documents can substitute complex static documents. Tools like sonarqube for static code analysis or jaeger, zipkin, ELK, prometheus/grafana and NewRelic for distributed monitoring and tracing services in production can give an accurate and real time view of code and its architecture.
Drinking from the stream. slides.
Streaming IoT Data and MQTT Messages to Apache Kafka.
DDD Ports and Adapters with Onion architecture, what goes where?
What is left inside a Hexagon is a logic to gather external data, call a decision maker and process result.
Layers, Onions, Ports, Adapters: it's all the same
I've put the UI components (the orange boxes) and the Data Access components (the blue boxes) in the same laye
DDD, Hexagonal, Onion, Clean, CQRS, … How I put it all together
the typical application flow goes from the code in the user interface, through the application core to the infrastructure code, back to the application core and finally deliver a response to the user interface.
while the CLI console and the web server are used to tell our application to do something, the database engine is told by our application to do something
The adapters that tell our application to do something are called Primary or Driving Adapters while the ones that are told by our application to do something are called Secondary or Driven Adapters.
Cockburn on Hexagonal Architecture
The ports and adapters pattern is deliberately written pretending that all ports are fundamentally similar. That pretense is useful at the architectural level. In implementation, ports and adapters show up in two flavors, which I’ll call ‘’primary’’ and ‘’secondary’’, for soon-to-be-obvious reasons. They could be also called ‘’driving’’ adapters and ‘’driven’’ adapters.
good description of ports and adapters
Asymmetry: Configurable Dependency implementation is different for each side. In the driver side, the application doesn’t know about which adapter is driving it. But in the driven side, the application must know which driven adapter it must talk to.
Isn't this just a layered architecture with a different name and drawn differently?. Onion vs. N-Layered Architecture. hexagonal architecture with spring data. video.
The Evolution of Comcast’s Architecture Guild
Application Integration for Microservices Architectures: A Service Mesh Is Not an ESB
Real-time Data Processing using Redis Streams and Apache Spark Structured Streaming
How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh. tweet
code with few possible control flow combinations, direct style (can always trace what connects to what), doesn’t violate grep test, comments explain why.
FRP can help increase code cohesion
Lessons Learned Replacing a DI Framework in a Legacy Codebase
DI composition root. What is a composition root in the context of Dependency Injection. more. more. more. more. more. Clean Composition Roots with Pure Dependency Injection (DI). Are We Just Moving the Coupling?
Microservices, Apache Kafka, and domain-driven design
91 global variables in Excel that were protected by one spin lock. No one could unravel the hairball
A service mesh is only meant to be used as infrastructure for communicating between services, and developers should notbe building any business logic inside the service mesh.
Restoring Confidence in Microservices: Tracing That's More Than Traces
The Potential for Using a Service Mesh for Event-Driven Messaging
Aggregating REST and Real-Time Data Sources
USE THE MOST PRODUCTIVE STACK YOU CAN GET. JAVA'S JOB LISTINGS, JWT, KAFKA, SERVERLESS, STREAMING, JARS IN WARS, THREADS, CODE COVERAGE--63RD AIRHACKS.TV
the state of Java relational persistence. slides. Spring Data JPA from 0-100 in 60 Minutes
TRANSACTIONS, J2EE, JAVA EE, JAKARTA EE, MICROPROFILE AND QUARKUS
regarding bad internal technology. HN.
Fast key-value stores: An idea whose time has come and gone. hn
Getting value out of your monad
the Challenges of Operationalizing Microservices
Mistakes we made adopting event sourcing
And if you store events with both an event_timestamp and effective_timestamp, you get bi-temporal state for free too. Invaluable when handing a time series of financial events subject to adjustments and corrections. For instance, backdate interest adjustments due to misbooked payments, recalculate a derivatives trade if reported market data was initially incorrect, calculate adjustments to your business end of month P&L after correcting errors from two months ago.
as time goes by - technical challenges of bi-temporal Event Sourcing. same talk. event sourcing with bi-temporal data
The evolution of the Shopify codebase
forging a functional enterprise
Feature Flags and Test-Driven Design: Some Practical Tips
Design Techniques for Building #Streaming Data, Cloud-Native Applications
Von Service-orientierten Architekturen (SOA) zu DDD und Microservices
Updating Materialized Views and Caches Using Kafka
Event-sourcing at Nordstrom: Part 2
How To Keep the Layers of your Spring App Separate using Integration Tests
MySQL CDC with Apache Kafka and Debezium
Message transformations for change data capture
Dependency Management and Versioning With a Maven Multi-Module Project
From PHP to transactions - airhacks
Package by feature or by layer
hexagonal architecture in practice more more
intimidated by the sheer breadth of #DDD
"Building audit logs with change data capture and stream processing"
you can't have a rollback button. Rolling Forward and other Deployment Myths
CDC, @debezium Streaming and @apachekafka an http://airhacks.fm episode
How many storage devices does a workload require?
Battle of the circuit breakers
Streaming Database Changes with Debezium by Gunnar Morling slides
Kafka Streams: Topology and Optimizations
How to sleep at night having a cloud service: common Architecture Do's
"Stop Mapping Stuff in Your Middleware" Logic in the database vs. logic in the application
The dark side of events. Finding your service boundaries. Monolith Decomposition Patterns. the usefulness of pre-allocating ids at the beginning. Event-Driven Microservices, the Sense, the Non-sense and a Way Forward
Kafka stream workshop and slides.
The Configuration Complexity Curse – Don’t Be a YAML Engineer
auto-formatting @java source code as part of the build process is a blessing
Have you built applications following #DDDesign principles, using #JPA for persistence?
Vertical Slices. Out with the Onion, in with Vertical Slices. The Importance of Vertically Slicing Architecture. Vertical Slice Architecture. APLICA VERTICAL SLICE. Why vertical slice architecture is better. Our architecture is a mess! Are you sure?
Ensuring rollback safety during deployments. Dealing with safely rolling forward and rolling back stateful services isn't something people talk about much, if at all. It's the sort of thing that gets hand-waved away.
Our setup of Prometheus and Grafana (as of the end of 2019)
A Thought Experiment: Using the ECS Pattern Outside of Game Engines. cache-friendliness
CSRF, XSS, JWT, REACTIVE DATABASES, TX AND WEBSOCKETS, JSON-B, OPENSHIFT
Practical Change Data Streaming Use Cases with Apache Kafka & Debezium
Qualities of a Highly Effective Architect
END-TO-END ARGUMENTS IN SYSTEM DESIGN
Modular Monolithic Architecture
The Let It Crash Philosophy Outside Erlang
Scaling to 100k Users. complexity
Data Modernization for Spring-based Microservices
Why did disabling hyperthreading make my server slower?
Testing Microservices: an Overview of 12 Useful Techniques - Part 1
This is not the class of software that I had in mind when I wrote the blog post 10 years ago. If your team is doing continuous delivery of software, I would suggest to adopt a much simpler workflow (like GitHub flow) instead of trying to shoehorn git-flow into your team.
If, however, you are building software that is explicitly versioned, or if you need to support multiple versions of your software in the wild, then git-flow may still be as good of a fit to your team as it has been to people in the last 10 years. In that case, please read on.
Branching is a core concept in Git, and the entire GitHub flow is based upon it. There's only one rule: anything in the master branch is always deployable.
Ready for changes with Hexagonal Architecture | Netflix Tech Blog
Re-architecting 2-tier to 3-tier
GOTO 19 "Good Enough" Architecture . Monolith Decomposition Patterns • Sam Newman Building Resilient Frontend Architecture • Monica Lent
Systems design for Advanced Beginners
humble guide to database schema design
Beyond the Distributed Monolith
Should services always return DTOs, or can they also return domain models?. Service layer returns DTO to controller but need it to return model for other services. Pass DTO to service layer. Map DTOs and Models within the Service Layer due to a missing Business Layer. Entity To DTO Conversion for a Spring REST API. LocalDTO (2004)
Not only you could pass DTO objects to Service Layer, but you should pass DTO objects instead of Business Entities to Service Layer.
Your service should receive DTOs, map them to business entities and send them to the repository. It should also retrieve business entities from the repository, map them to DTOs and return the DTOs as reponses. So your business entities never get out from the business layer, only the DTOs do.
Some people argue for them as part of a Service Layer API because they ensure that service layer clients aren't dependent upon an underlying Domain Model. While that may be handy, I don't think it's worth the cost of all of that data mapping. As my contributor Randy Stafford says in P of EAA "Don't underestimate the cost of [using DTOs].... It's significant, and it's painful - perhaps second only to the cost and pain of object-relational mapping".
Let's now look at a service level operation – which will obviously work with the Entity (not the DTO)
@GetMapping @ResponseBody public List getPosts(...) { //... List posts = postService.getPostsList(page, size, sortDir, sort); return posts.stream() .map(this::convertToDto) .collect(Collectors.toList()); }
Enterprise applications typically require different kinds of interfaces to the data they store and the logic they implement: data loaders, user interfaces, integration gateways, and others. Despite their different purposes, these interfaces often need common interactions with the application to access and manipulate its data and invoke its business logic. The interactions may be complex, involv-ing transactions across multiple resources and the coordination of several responses to an action. Encoding the logic of the interactions separately in each interface causes a lot of duplication.
[ON DTOS (old)] much conflicting information!
DTOs are only created when their structure significantly differs from the that of the entity. In all other cases the entity itself is used. The cases when you don’t want to show some fields (especially when exposing via web services to 3rd parties) exist, but are not that common. This can sometimes be handled via the serialization mechanism – mark them as @JsonIgnore or @XmlTransient for example
Don’t use the mappers/entity-to-dto constructors in controllers, use them in the service layer. The reason DTOs are used in the first place is that entities may be ORM-bound, and they may not valid outside a session (i.e. outside the service layer).
performance of Java mapping frameworks
Creating large Java applications composed of multiple layers require using multiple models such as persistence model, domain model or so-called DTOs. Using multiple models for different application layers will require us to provide a way of mapping between beans.
Dozer is a mapping framework that uses recursion to copy data from one object to another. The framework is able not only to copy properties between the beans, but it can also automatically convert between different types.
sooo subsetting and type conversions are an important part of mapping frameworks?
The magic behind the Dependency Injection of Quarkus
Java & SQL - stronger together overview of the lasagna
Domain Events Versus Change Data Capture
From Batch to Streaming to Both. Internet of Tomatoes: Building a Scalable Cloud Architecture.
Software Architecture for Agile Enterprises
Event-Driven Architectures for Spring Developers
Adoption of Cloud Native Architecture, Part 2: Stabilization Gaps and Anti-Patterns
Things I Wish I’d Known About CSS
Dividing front end from back end is an antipattern (?)
Domain-Oriented Microservice Architecture
For any event-based system, the message structures it exposes to external consumers are its public interface. Evolve their schemas with the same care and attention to backwards compatibility as your synchronous APIs.. cdc breaks encapsulation
How to design a REST API that can “prompt” the client about long-running operations?
Building dashboards for operational visibility
Inside the Hidden World of Legacy IT Systems.
I've tackled legacy systems my entire career, and there is a certain art to untying the knot of dependencies, procedures, and expectations.
It’s painful to see the people who know all the hotkeys and key sequences on an old green terminal suddenly thrust into a world of mouse hunting and clicking. It makes you wonder if new things are really better.
Under Deconstruction: The State of Shopify’s Monolith
To Microservices and Back Again very good.
Design Microservice Architectures the Right Way (2018)
Event Sourcing You are doing it wrong (2018) See the papers mentioned at the end: the dark side of event sourcing. versioning in an event sourced system.
Moving BBC Online to the cloud
Go in Production – Lessons Learned
Asynchronous Task Scheduling at Dropbox
If you have the opportunity, please do not build it like this. Referring to the architectural diagram, it is going to be much more efficient for the "Frontend" to persist the task data into a durable data store, like they show, but then the Frontend should simply directly call the "Store Consumer" with the task data in an RPC payload. There is no reason in the main execution path why the store consumers should ever need to read from the database, because almost all tasks can cut-through immediately and be retired. Reading from the database should only need to happen due to restarts and retries of tasks that fail to cut through.
One of the best, and first, things we did when starting our machine learning platform was to design it using a plugin architecture. There's a lot of scar tissue and horrible experience through our previous ML products we built for enterprise. Namely, it was extremely hard to onboard new developers to work on the product. They had to understand the whole thing in order to contribute.
Not Just Events: Developing Asynchronous Microservices. Creating event-driven microservices: the why, how and what.
Stored Procedures as a Back End
sagas - Azure reference architectures. sagas for consistency. 2. Not Just Events: Developing Asynchronous Microservices. Battle-tested event-driven patterns for your Microservices archit. Opportunities and Pitfalls of Event-driven Utopia
Clean Architecture Boundaries with Spring Boot and ArchUnit
Clean Architecture with Spring by Tom Hombergs
If All You Have Is a Database, Everything Looks Like a Nail. HN.
Soon, there was an established trend that increased the entropy and intertwining of applications and tables. It became common to have transactional updates across tables for different apps.
Sometimes people stage read-only copies of tables. These are asynchronously updated from the authoritative owning application. Other applications then “own” the read-only copy in their application set of tables.
is only good advice if the tables are application specific data and you don't do microservices in that stupid braindead way that makes it so that everything from the admin panel to data visualization are their own "applications" with their own databases and doing things that would be even the simplest of queries becomes a project in writing what are effectively bad performance joins via random http APIs. IE have a data model and understand where the most painless boundaries are, don't throw up dozens of DBs for the hell of it.
Look into the patterns of CQRS, event sourcing, flow based programming and materialized views. GraphQL is an interface layer, but you still have to solve for the layer below. API composition only works when the network boundary and services are performance compatible to federate queries. The patterns above can be used to work around the performance concern at a cost of system complexity.
Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, so all the HTTP calls are mocked and the responses are meaningless!
A lot of posters here seem to have been deeply burned from microservices designed along the wrong lines. I mean, sure, it happens. You're going to make mistakes just like you can misjudge how to separate concerns in a set of classes. It shouldn't be an issue to fix it. Maybe some teams focus on pure separation before they have a solid design? Maybe its just a culture of feeling like service boundaries can never change?
There’s a model of software based around shipping events around, and subscriptions between systems. The purposes of separation are at least a couple important, perhaps you know. Each has a DB, often embedded, that is suitable and materialized from the subscriptions and its own; mutated predictably.
Software Design for Flexibility book.
But in practice, the accumulation of cold data on a local disk is where this starts to hurt, particularly if that has to serve read traffic which starts from the beginning of time (i.e your queries don't start with a timestamp range).
KSQL transforms does help reduce the depth of the traversal, by building flatter versions of the data set, but you need to repartition the same data on every lookup key you want - so if you had a video game log trace, you'd need multiple materializations for (user) , (user,game), (game) etc.
- Write a event recording a desire to checkout. 2) Build a view of checkout decisions, which compares requests against inventory levels and produces checkout results. This is a stateful stream/stream join. 3) Read out the checkout decision to respond to the user, or send them an email, or whatever.
CDC is great and all, too, but there are architectures where ^ makes more sense than sticking a database in front.
Admittedly working up highly available, stateful stream-stream joins which aren't challenging to operate in production is... hard, but getting better.
Unpopular opinion: SQL is better than GraphQL. some good aspects. better for trees and DAGs. level limitation. what about using views?. another (older) comparative
GraphQL is better when what you are requesting is best expressed as a tree (or a "graph", though only the DAG variety). This is not always the case, but it very often is when building API:s for production use.
Of course, you can express tree structures in table form, but it is not very convenient for clients to consume. In particular if your client is rendering nested component views, what you want is very often something hierarchical.
performance is more predictable, exactly because the language is more restricted. You can't just join in all the things, or select billions of rows by accident. The schema dictates what is allowed.
you can't request a tree structure (eg: a menu with submenus) with an unknown number of levels.
You don't have to expose your entire schema, instead expose carefully designed SQL views (so you can refactor your tables without breaking your API)
Lessons Learned from Reviewing 150 Infrastructures
sharing transactions and persistence contexts across module boundaries -- yea or nay?
Data architecture vs backend architecture
Bit little guide to message queues.
How we rebuilt the Walmart Autocomplete Backend
React created roadblocks in our enterprise app. original link. Using react in enterprise contexts. The "seams" link in that last one is interesting as well.
Software engineering topics I changed my mind on
reworking of GHC's errors nice architectural choice to avoid cyclic dependencies
The complexity that lives in the GUI
The Database Inside Your Codebase
You probably don’t need a micro-frontend
Developing microservices with aggregates
Modules, monoliths, and microservices. hn.
Why isn't Godot an ECS-based game engine? . lobsters.
Capturing Every Change From Shopify’s Sharded Monolith
Backpressure in Reactive Systems
necessarily microservices but something akin to serverless functions running on a managed platform
Software Architecture Design for Busy Developers
The pedantic checklist for changing your data model in a web application
database migrations and continuous delivery
Zero-downtime schema migrations in Postgres using views
Notes on streaming large API responses
don't forget structure and then try to remember it
Microservices and Cross-Cutting Concerns
Qualities of a Highly Effective Architect
The Database Ruins All Good Ideas
Thinking in Events: From Databases to Distributed Collaboration Software
On the Evilness of Feature Branching
How much business logic should be allowed to exist in the controller layer?. How accurate is “Business logic should be in a service, not in a model”?. Why put the business logic in the model?.
Soliciting requirements is a iterative process, starting at an abstract level and diving down as you iterate. It is a data pull from the stakeholders; so it is about asking a ton of questions, several different ways, and becoming more tactical as you go along.
Solving the double (quintuple) declaration Problem in GraphQL Applications
Domain services (2012). Services in Domain-Driven Design (DDD).
application services which act as a facade. Application services are simple classes which have methods corresponding to use cases in your domain
When a significant process or transformation in the domain is not a natural responsibility of an ENTITY or VALUE OBJECT, add an operation to the model as standalone interface declared as a SERVICE. Define the interface in terms of the language of the model and make sure the operation name is part of the UBIQUITOUS LANGUAGE. Make the SERVICE stateless.
Retry long-running message processing in case of processing node failure.
Are Repositories implementations part of my domain? Should repositories have SQL queries?
