How does Axon work with State Stored Aggregate, Database Transaction, Message Dispatching, and Message Storing? What happens in case of a failure between these operations? Is there some kind of inbox and outbox mechanism?
Thanks!
Hi,
I found this forum topic The outbox pattern which ends up in an issue Consistency gap in conjunction with state-stored aggregates · Issue #1103 · AxonFramework/AxonFramework · GitHub.
Please don’t get me wrong but I can’t see why this is not a priority. I’ll try to explain the reason:
In a “traditional” approach we have database transactions that allow us to change multiples entities/aggregates with the price of long blocking time and high memory consumption. Reads and writes are done in the same database too what makes everything even harder.
At some point in time, the application cannot handle all the requests using this architecture. So as most specialists recommend instead of switching straight to event sourcing we could split the application into reads and writes and make communication between entities/aggregates through events. But, because of the open issue, in my point of view, this is not an option. We can end up in a real inconsistency state which it’s not possible to recover unless finding the broken pieces and fix them manually but if we’ve moved to this architecture it means that the scale does not permit fix them manually. Imagine a stock management system or an accounting system in the cloud with multiple customers…
If I’m wrong, please help me see it.
Thanks!
in order to solve this we have introduced a local event store which persists all the events within a local transaction first.
Then there is a component which picks up the events and publishes them to the global event store.
The idea is similar to the outbox pattern.
Nevertheless, there is no support from Axon Framework itself (so far).
To be honest, we decided to move away from State-Stored aggregates to Event-sourced aggregates as it offers more advantages compared to the slightly increased efforts.
Oliver
1 Like
Hi @Oliver_Libutzki,
Thanks for sharing your experience.
Your case confirms my point. What type/category of software do you work on?
@moderators Would you have something to add or clarify? Huge thanks!
Is there a reason to not have a local event store with all events?
We currently have a global event store (in house design with which I’m involved), but we’re considering converting the event store to a module that can be integrated to avoid the outbox pattern that we’re using.
So we are currently thinking of moving the event store to the same database, and having consumers contact the service producing those events directly to get an event stream. This will introduce additional load on the producing service, but our thinking is that if this becomes a problem we could always consume all events in a replication service which can then serve these to other consumers.
I’m late to the party here, but since I promised @Murilo_Carlos_Cardos to give my 2 cents on the topic, here they are 
State-stored aggregates are the closest to the traditional way of doing things. Since the first app I wrote (almost 30 years ago), the “how do we persist the data?” has been among the top concerns. The most crucial piece of every software I was ever involved with was the database and/or the persistence framework. The concept of “we must store the state” is so deeply rooted in most developers’ experience that it is next to impossible to let go of it. I was taught and trained to think this way, and it took me a long time to understand and accept the alternative.
There is nothing wrong with storing state per se. It works perfectly fine in a single deployable unit using a single data source. Even multiple deployable units connected to the same single data source work fine, assuming the data source provides proper transaction isolation. The moment you have multiple data sources, things become exponentially more complex. You have two options:
- take the “stop the world” approach and wait for a “distributed transaction” to complete
- embrace eventual consistency with all of its benefits and drawbacks
Looking at it this way, I tend to disagree with suggestions like this one:
Switching to CQRS (which is what the above statement describes) means you have at minimum two data sources (one for the write/command side and one or more for the read/query side). Applying the “stop the world” approach hardly buys you anything in such a scenario. Thus you move to the land of eventual consistency. The biggest challenge in this land is to have a source of truth. Otherwise, as you state it yourself:
Having more than one data source, each claiming a different thing and no way to know what the truth is, is the worst possible situation you can be in. Many solutions (including the outbox pattern mentioned earlier) focus on not letting different sources claim different things. I don’t think they can always guarantee that unless they apply the “stop the world” approach. I think it’s by far more critical to know what the truth is. You can correct the data and recover from any inconsistency if you do. And the truth is in a central, immutable log containing every significant event in the system.
When you have that, you can always recover, which does not mean it is a straightforward process. As no changes occur on the query side, it may be as easy as dropping all the data and regenerating it from events. On the command side, an inconsistency may lead to a potentially long chain of undesired events that you need to somehow compensate for. The only truly reliable way to avoid that is to make sure aggregates operate on data from a single source of truth. State-stored aggregates can not assure that as they depend on other infrastructural components (persistence layers, databases, etc.), which may fail.
So I can’t provide an official answer to why Consistency gap in conjunction with state-stored aggregates · Issue #1103 · AxonFramework/AxonFramework · GitHub is not a priority. Perhaps @Steven_van_Beelen can. But given everything I’ve written above, I agree it shouldn’t be. IMHO, it focuses on the wrong solution (or at least one outside the framework’s main scope) to the inconsistency problem - not letting different sources claim different things. Meanwhile, the framework already provides a better solution for that problem - event log and event-sourced aggregates.
1 Like
I couldn’t put it better, Milen. As time progresses and users of the Framework increase, we feel more and more confident that Event Sourcing is, 9 out of 10, the way to go.
Nonetheless, there are and always will be exceptions to this case.
Hence why the ticket isn’t closed, but still under discussion.
And, also why it hasn’t been prioritized yet.
Note, that at all times, Axon Framework still is an open-source, collaborative, project. If someone really (really) needs a solution for #1103, providing a pull request would be the easiest way to increase the priority at this moment. Right now, we simply don’t have the manpower to delve deeper into this, even though in my ideal world we’d have no open issues at all.
First of all, thank you very much for the answers @Steven_van_Beelen @milendyankov
I understand all the points of view and I just want to point out two things:
-
@milendyankov mentioned eventual consistency and my question was about real inconsistency. If there is no mechanism such as inbox and outbox that guarantee the consistency between the database and messaging infrastructure, who decides to go for the State Stored Aggregate approach is exposed to a real inconsistency scenario. In this scenario where something bad happens between the operation in the database and the messaging infrastructure (lost or phantom messages), just can be consistent again by manual manipulation.
-
@Steven_van_Beelen I completely understand that it is critical to focus on more important things first but it does not invalidate the technical gap.
I completely agree with you here, Murilo.
I do hope my point didn’t come across as such.
By no means did I intend to state this predicament is not a valid concern to cover with Axon Framework (albeit it, in the future).
1 Like