Beiträge

I went to a great session about CQRS, Event Sourcing and domain-driven Design (DDD) on the Software Architecture Summit. The speaker Golo Roden (@goloroden) did a fantastic job selling these concepts to his audience with a great storytelling approach. He explained why CQRS, Event Sourcing and DDD fit together perfectly while replicating the www.nevercompletedgame.com for his daughter. This is what he shared with us.

Domain-driven Design

The more enterprise-y your customer the weirder the neologisms get.

We – as software engineers – struggle to understand business and domain experts. Once we understand something, we try to map it to technical concepts. Understood the word "ferret"? Guess we need a database table called "ferret" somehow. We then proceed to inform our business colleagues, "deploying a new schema is easy as we use Entity Framework or Hibernate as OR-mapper". He thinks we understood, we think he understood. Actually, nobody understood anything.
As software engineers we tend to fit every trivial and every complex problem into CRUD-operations. Why? Because its "easy" and everyone does it. If it was that easy, software development would be effortless. Rather trying to fit problems in a crud pattern, we should transform business stories into software.
That’s why we should use domain-driven design and ubiquitous language.
Golo Roden proceeds to create a view on the nevercompletedgame with ubiquitous language. So nobody asks, "what does open a game mean" and there is no mental mapping.
I won’t go into detail here, but an example can show why we need this.

  • Many words have one meaning: When developing a software for a group of people, sometimes we call them users, sometimes end-users, sometimes customers etc. If we use different words in the code or documentation and developers join a project later – they might think there is a difference between these entities.
  • One word has many meanings: Every insurance software has "policies" somewhere in its system. Sometimes it describes a template for a group of coverages, sometimes it’s a contract underwritten by an insurance, sometimes a set of government rules. You don’t need to be an expert to guess this can go wrong horribly.

CQRS

Asking a question should not change the answer

Golo Roden jokes, "CQRS is CQS on application level", but actually it’s easy to understand this way, once you read a single article about CQS. Basically, it’s a pattern where you separate commands (writes) and queries (reads): CQS.

  • Writes do not return any values and change the state of an object.
    stack.push(23); // pushes value 23 onto the stack; returns nothing
  • Reads return a value and don’t change the state.
    stack.isEmpty() // does not change state; returns a isEmpty boolean
  • But don’t be fooled! Stacks are not following the CQS pattern.
    stack.pop() // returns a value and changes state

Separating them on application level means: exposing different APIs for reading (return a value; do not change state) and writing (change state; do not return value *). Or phrased differently: Segregate responsibilities for commands and queries: CQRS.

* for http: always returns 200 before doing anything

Enforcing CQRS could have this effect on your application:

For synchronizing patterns see patterns like the saga pattern or 2 phase commit . For more reference see: Starbucks Does Not Use Two-Phase Commit

Event Sourcing

When talking about databases (be it relational or NoSQL) often we save the current state of some business item persistently. When we are ambitious we save a history of these states. Event sourcing follows a different approach. There is only one initial state, change requests to this state (commands) and following manipulating operations (events). When we want to change the state of an object, we set up a command. This triggers an event (that’s published so some kind of queue) and most likely is persistent in a database.

Bank account example: we start with 0 € and do not change this initial value when we add or withdraw money. We save the events something like this:

Date EventId Amount Message
2019-01-07 e5f9e618-39ad-4979-99a7-342cb1962266 0 account created
2019-01-11 f2e98590-7795-4cf7-bdc2-1794ad39874d 1000 manual payment received
2019-01-29 cbf44bfc-7a5e-4514-a906-a313a6e0fb5e 2000 saylary received
2019-02-01 32bc638c-4783-45b8-8c1e-bebe2b4528a1 -1500 rent payed

When we want to see the current balance, we read all the events and replay what happened.

const accountEvents = [0, 1000, 2000, -1500];
const replayBalance = (total, val) =>  total + val;
const accountBalance = accountEvents.reduce(replayBalance);

Once every n (e.g. 100) values we save a snapshot to not have to replay too many events. Aside from the increased complexity this has some side effects which should not be unadressed.

  • As we append more and more events, the data usage increases endless. There are ways around removing "old" events etc. and replacing them with snapshots, but this destroys the intention of the concept.
  • Additionally, as more events are stored, the system gets slower as it has to replay more events to get the current state of an object. Though, snapshotting every n events can get deterministic maximum execution time.

While there are many contra arguments there is one the key benefit why its worth: your application is future proof, as you save "everything" for upcoming changes and new requirements. Think of the account example from the previous step. You can implement/analyze everything of the following:

  • "How long does it take people to pay their rent once they got their salary"
  • "How many of our customers have two apartments? How much is the difference between both rents?"
  • "How many of our customers with two apartments with at least 50% in price difference need longer to pay off their car credit?"

To sum it up and coming back to our initial challenge, our simple CRUD application with domain-driven Design, CQRS and event sourcing would have transformed our architecture to something like this:

While this might solve some problems in application and system development this is neither a cookie-cutter approach nor "the right way" to do things. Be aware of the rising complexity of your application, system and enterprise ecosystem and the risk of over-engineering!

Surely, you’ve heard the fairytales from microservices and monoliths. Or on a similar note, the tales about distributed (big) balls of mud from people like Simon Brown.


Usually these posts point out what goes wrong and how unexperienced teams go for a "hype-driven/tunnel-vision architecture". But how do you actually cut microservices? How do you design interfaces? What are techniques to find weak points in your application or system architecture?
In this post I digest the intends and views of a talk on the software architecture summit in Munich.

Speakers

Herbert Dowalil @hdowalil on Twitter
Stefan Zörner @STefanZoerner on Twitter

Microservices vs. Monolith

The spectrum of architecture definitely isn’t as binary as most blog posts suggest it is. There are way more types, here is a short recap of some of the most famous ones.

Microservices

There are different definitions out there, but most of them share key points like a domain-driven modules cut, outstanding flexibility and network distribution. One (generally accepted) definition in a FAQ-style by Jimmy Bogard.

Self-contained Systems (SCS)

Similar to microservices, but usually bigger services and fewer in a system.

Deployment Monolith(aka. Modulith)

You have a valuable product, but it does not suite a SaaS business model? Most of your customers won’t offer a Kubernetes clusters or you have a hard time building a secure deployment pipeline? This does not stop good architecture! You can still cut your software in great modules and deploy it as one package.

Architecture Monolith

Such a monolith has no defined structure. It consists of modules, but they are referenced across the whole system. Looks like this:

SOA

A blueprint of doing application architecture to generate standard services showing actual business processes steps. The main benefit should’ve been the orchestration of services to full processes and composition of new processes.
While SOA might have failed for most, it is "survived by its offsprings" (Anne Thomas Manes: SOA is Dead; Long Live Services). For a comparison between SOA and microservices see this O’REILLY report.

All these types focus on cutting a big software block into modules. Microservices for example focus on the smallest autonomous boundaries, SOA on reusability and composition. Because of this, the following ideas ‘how to define modules’ can be used for microservers, but don’t stop there. You can use them to define Java packages, C++ namespaces or create C# assemblies.

LET’S START

Often we start with question groups like "How many parts do I need?" and "Where do I cut?"

Sometimes we don’t even know if we should initially start with a monolith (Martin Fowler: MonolithFirst) or with microservices (Stefan Tilkov: Don’t start with a monolith). We can answer this particular question now: it doesn’t really matter when designing modules and interfaces. When you need a push for microservices think about upcoming complexity. The speakers phrased it nicely:

Modularization makes complexity manageable. Investment at beginning makes complexity manageable at the end.

For the first questions, we follow an enhanced version of the SOLID criterias by Robert C. Martin. Herbert Dowalil calls them "5C". Such principles sound nice, but what does strong cohesion or lose coupling mean?

"If you can’t measure it – you can’t manage it"

The key here is using "old school" metrics like cyclomatic complexity or average relative visibility. That’s it; here are the 5C:

Cut

Cut your modules in a way where they only have one concern/a single responsibility. Try to maximize cohesion inside a module.
Metrics: e.g. LCOM4, Relational-Cohersion, cyclomatic complexity

Conceal

Hide the internals of a module inside. Nothing outside the module needs to know about the technologies used or any work (changes) done inside.
Metrics: e.g. low relative visibility, low average relative visibility and low global relative visibility

Contract

Design small, specific and easy to understand interfaces between modules.
Metrics: e.g. Depth-of-Inherence

Connect

Explicitly declare every connections between modules.
Metrics: e.g. RACD and NCCD from John Lakos. Stability from the software package metrics.

Construct

Build new modules by connecting modules together. Move from a lower level of modules to a bigger view on the system. For higher level, modules follow the same patterns as for lower level ones.
Metics: e.g. automated tools like ArchUnit, Sonorgraph or Teamscale

For a deeper dive, see the "Architektur Spicker" (GER only) and follow @Herbert Dowalil on Twitter for updates on his new book.

Once you’ve cut your modules you can decide if you want to distribute them over the network. Pros can be:

  • Independent technological decisions:
    This reduces macro architecture from your software and lets you use specialized tools and languages for each module (like .NET for a web API and python for the underlying data science or spring boot for your user handling and c++ for a calculation core).
  • Technology roll-over:
    You can upgrade your technology components step by step. An upgrade from the Java or Node.js runtime might improve performance and eliminate security threads or you can change a module from an outdated version to the newest one (Angular.js to Angular). This can be useful when a system runs longer than anticipated.
  • Developer autonomy:
    Every part of your software can be developed and deployed independently. This shortens feedback cycles and enables developers to evaluate ideas faster.

While this sounds awesome (and it is!), there are also downsides:

  • Troubleshooting and debugging:
    Following calls and processes through your distributed software is harder than debugging a single component. Tracing makes troubleshooting easier, but replicating an error state across your whole application can be hard.
  • Complicated Ops:
    Operating your software can be way harder. For example, there might be issues with creating secure pipelines and authorization concepts through company boarders.
  • Consistency:
    There are ways to build around eventual consistency and distribution and sacrifice strong consistency. Examples would be the 2 phase commit (but you shouldn’t) or the saga pattern. While these are working and are proven successful, the initial set up is harder and they certainly don’t simplify troubleshooting.

As usual, this trade-off needs to be evaluated individually.

Take-aways

Modularization is hard. If you decide to go with microservices or any other way, doesn’t really matter in this context. You can distribute your modules to enforce principles, but if you cut your modules wrong, things will only get harder.
Don’t be afraid of "old school" or "university-style" metrics. Identify metrics with your team. Collaboratively search for weak points in your software (architecture). Enforce selected metrics, but never let a metric break the build and never dictate metrics top down!

LITERATURE and FOLLOWUP READING

Das Microservice-Praxisbuch by Eberhard Wolff

Arc42

Software systems architecture by Nick Rozanski and Eoin Woods

Microservice Patterns by Chirs Richards

[Modulith First Der angemessene weg zu microservices]() by Herbert Dowalil

Architektur Spicker #8

Microservice FAQ by Jimmy Bogard

SOA is dead; long live services by Anne Thomas Manes

What distinguishes a junior developer from a senior and the senior from a software architect? This is a commonly asked question and there are plenty of very good sources out there. One argument can be found on every blog post about this topic: a junior mostly makes small decisions and consumes knowledge. With the level of seniority, the level of decision making and knowledge sharing increases. This is why we often find people like "advocates", "fellows" or "heroes" on conferences and summits.
I went to one of these summits (Munich Software Architecture Summit) and want to share my experiences about the sessions and talks here. Let’s start with the key note of the first day. Weiterlesen