Advanced application architecture

0-refresher.md

Refresher on anti-patterns and best practices

Most important basic philosophy “simple vs. easy”: Rails Conf 2012 Keynote: Simplicity Matters by Rich Hickey (Slides)

You probably know this already and in Ruby not all of it is done “explicitly” but it’s important to have these principles in mind when getting to know the following architecture concepts.

STUPID: Singleton, Tight Coupling, Untestability, Premature Optimization, Indescriptive Naming, Duplication
SOLID: Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, Dependency Inversion

A fairly good summary: From STUPID to SOLID Code

Why not just use Rails?

Because “the Rails way” leads you to violate basically all of the principles mentioned above (and more) which inevitably ends up in a “big ball of mud”.

In-depth explanation of application engineering problems and solutions: Ruby Midwest 2011 - Keynote: Architecture the Lost Years

Key takeaway: “MVC is not an architecture.” Instead, we need a real application architecture.

1-application-architecture.md

Application architecture

In the above talk, Bob Martin draws up something that is very close to “Hexagonal Architecture”, also known as “Ports & Adapters”. Here’s more details and visualizations: Hexagonal architecture

The idea is to have a core application that contains business logic and cares about little else.
Every interaction with the outside world is modeled as a “port” in which you can stick one of multiple “adapters”.

Examples for such ports & adapters are:

• Actions can be triggered via a website, web API, command-line interface, events read from a message bus, …
• Data can be retrieved via a website, web API, command-line interface, events posted to a message bus, …
• Data can be stored in memory, on the filesystem, in a local SQLite database, in a remote PostgreSQL database, …

Side note: This makes Rails (or Padrino) a viable option as a web adapter, and a great one! But due to the underlying mindset of Rails (or any MVC framework) you must be very careful to stay decoupled from it – which takes effort.

2-domain-engineering.md

Domain engineering

Having a solid architectural foundation, we can tackle the “real” problem: domain engineering. This means we use the knowledge about our business domain to solve exactly the needs of our users. To do that we focus around contexts, processes, entities, and events.

Application engineering then becomes the implementation of our domain design. The following techniques are immensely helpful to achieve a good implementation of domain-specific solutions to domain-specific problems.

DDD

Stands for “domain driven designed” and is based on the idea of building for a particular domain (in our case advertising agencies).

Within that domain you usually have a ubiquitous language when your domain experts talk about objects, processes, events, etc. However, in several bounded contexts the same word might mean a different thing (e.g. in monitoring, optimization and billing we have a campaign, but we care about completely different aspects of it).

Presentation with a halfway good introduction: Domain Driven Design

3-application-engineering.md

Application engineering

DDD

Principles to keep in mind:

1. Keep architectural levels separated:

• Domain encapsulates business logic. (hexagonal architecture: the core)
• Application layer provides coordination, glue between domain and infrastructure. (hexagonal architecture: layer around core with ports)
• Infrastructure encapsulates outside world, e.g. interactions, persistence, third-party services. (hexagonal architecture: the adapters)

2. Keep contexts separated. Contexts can interact with each other through application-level services.
3. You don’t mess around with individual objects. Data is modeled as aggregates consisting of entities.

• Entities encapsulate related value objects and contain business logic to change or validate them. Example: client has a name and AdWords ID.
• Value objects are immutable! Example: To rename a client, you don’t modify its name, you give them a new name.
• Aggregates form the models you care about. Example: In monitoring we primarily care about the campaign = aggregate. A campaign then internally has performance data.
• You only interact with the entire aggregate through the aggregate root (the “primary entity” of the aggregate).
• ”Tell, don’t ask” – You don’t modify the aggregate directly, instead you ask the aggregate root to do something. The aggregate always stays in a consistent, valid state.

4. You don’t pull aggregates out of thin air. You fetch and store them using a repository. Only entire aggregates can be fetched/stored, not parts of them.

CQRS

Stands for “command query responsibility segregation”. The core idea is that we want to build task-based UIs instead of a stupid CRUD interface. We therefore look at “actions a user can take” and model them with commands, as well as “things the user can see” and model them with queries. Commands and queries are built individually for their respective purpose. All they have in common is that they live in the same domain context.

The responsibility segregation:

• On the “write side”, commands go through the domain and interact with aggregates to create/change them.
• On the “read side”, queries don’t go through the domain, they can directly pull whatever data is necessary from persistence.

Domain model vs. read model:

• On the “write side” you need a consistent domain model for all commands to use.
• On the “read side” you (a) don’t need to use the same model as in the domain and (b) don’t even need to use a common model for all queries.

This means: You can take a shortcut and use the domain model for your queries, but if your needs on the read side differ from the domain model you create a dedicated read model. If your needs differ between queries, you create a dedicated read model for each.

ES

Stands for “event sourcing” and is based on the idea to store each change as an individual event. Instead of building a CRUD application that modifies a single state of data, build an application that

1. applies commands to aggregates,
2. records any resulting changes with events,
3. then uses those events to restore (“replay”) the state of an aggregate
4. as well as to read whatever amount and structure of information in a query.

Important: ES has many desirable qualities but is also highly demanding. We won’t be able to implement (and benefit) from it until everything else is in place. To learn more about CQRS and ES, watch this talk: Greg Young - CQRS and Event Sourcing - Code on the Beach 2014
Quote: “You can use CQRS without ES, but if you use ES you must use CQRS.” That’s why we introduce CQRS first and later add on ES.

Article about combining those principles, with diagrams: Introduction to Domain Driven Design, CQRS and Event Sourcing
Tutorial applying those principles, with code samples: Edument CQRS and Intentful Testing Starter Kit
FAQ on lots of different aspects: Edument CQRS Entire FAQ

Good presentation on how to tackle all of this in a project: Brownfield Domain Driven Design

4-testing.md

Testing different parts of the system

With so many different layers and aspects of our application, testing becomes more than behavior-driven acceptance tests that go through the UI all the way to the database and test the application code more as a side effect.
Instead, we write specific types of tests for each part of our application, ranging from domain- and application-level unit tests, infrastructure-level integration tests, to component tests of the entire system.
As a result, we combine both TDD and BDD with a “test first” mindset.

Great summary on testing strategies: Testing Strategies in a Microservice Architecture
That site also gives an outlook on the implications of moving from a monolithic system to a distributed multi-service/micro-service architecture.

Good read on keeping tests simple: 7 Reasons I’m Sticking With MiniTest and Fixtures in Rails

Great but long series of conversations between Martin Fowler, Kent Beck, and Heinemeier Hansson on the principles of testing, testability, and “test-induced damage”: Is TDD Dead?