It’s very powerful whenever your event store is seperated from the persistence unit of the component which emits the event. There are many Axon use.cases:
Axon Server, Kafka, AMQP,…:
Separate JPA Persistence Unit in a monolith: The emitting component is managed by a different EntityManager than the event store (this is my concrete use case)
With the outbox pattern you can circumvent a lot of potential problems concerning atomicity.
The outbox stores the event which should be emitted. This outbox entry is committed within the transaction which also manages the component which emitted the event. This approach ensures that all data changes AND the event are stored in a single transaction.
A tracker checks for messages in the outbox, publishes the events to the event store and deletes the outbox entries afterswards.
As the deletion of the outbox entry might fail, it might happen that the same outbox entry is published to the event bus multiple times. Therefore the even bus has to be idempotent.
Maybe I did not get the concepts of Axon entirely, but currently I think something like the outbox pattern is missing. I’m keen to know your opinion.
Imho especially state based aggregates might benefit from the outbox pattern. Using event sourcing might circumvent this issue as you don't change any persistent data while processing a command.
Are there any chances/plans to use the outbox pattern in Axon?
Hello, I’m quite new to the axon framework but from my perspective what axon tries to focus on is Event Sourcing and CQRS, the ES is an alternative of your outbox pattern so you check that out.
W dniu poniedziałek, 6 maja 2019 22:21:08 UTC+2 użytkownik oli...@libutzki.de napisał:
It’s very powerful whenever your event store is seperated from the persistence unit of the component which emits the event. There are many Axon use.cases:
Axon Server, Kafka, AMQP,…:
Separate JPA Persistence Unit in a monolith: The emitting component is managed by a different EntityManager than the event store (this is my concrete use case)
With the outbox pattern you can circumvent a lot of potential problems concerning atomicity.
The outbox stores the event which should be emitted. This outbox entry is committed within the transaction which also manages the component which emitted the event. This approach ensures that all data changes AND the event are stored in a single transaction.
A tracker checks for messages in the outbox, publishes the events to the event store and deletes the outbox entries afterswards.
As the deletion of the outbox entry might fail, it might happen that the same outbox entry is published to the event bus multiple times. Therefore the even bus has to be idempotent.
Maybe I did not get the concepts of Axon entirely, but currently I think something like the outbox pattern is missing. I’m keen to know your opinion.
Kind regards
Oliver
W dniu poniedziałek, 6 maja 2019 22:21:08 UTC+2 użytkownik oli...@libutzki.de napisał:
It’s very powerful whenever your event store is seperated from the persistence unit of the component which emits the event. There are many Axon use.cases:
Axon Server, Kafka, AMQP,…:
Separate JPA Persistence Unit in a monolith: The emitting component is managed by a different EntityManager than the event store (this is my concrete use case)
With the outbox pattern you can circumvent a lot of potential problems concerning atomicity.
The outbox stores the event which should be emitted. This outbox entry is committed within the transaction which also manages the component which emitted the event. This approach ensures that all data changes AND the event are stored in a single transaction.
A tracker checks for messages in the outbox, publishes the events to the event store and deletes the outbox entries afterswards.
As the deletion of the outbox entry might fail, it might happen that the same outbox entry is published to the event bus multiple times. Therefore the even bus has to be idempotent.
Maybe I did not get the concepts of Axon entirely, but currently I think something like the outbox pattern is missing. I’m keen to know your opinion.
Kind regards
Oliver
W dniu poniedziałek, 6 maja 2019 22:21:08 UTC+2 użytkownik oli...@libutzki.de napisał:
Robert is right in that ES is our focus. We do support state stores aggregates, and in those cases, this pattern could help.
We have in fact explored this pattern in the early days of Axon 3. The biggest challenge was that you loose the abiluty to do consistency checks on your events prior to publishing. Also, it adds a lot to the latency of messages.
We decided back then to focus on the event sourcing approach first, where this pattern is not necessary (and even prohibitive).
There are a lot of caveats with the outbox pattern when it comes to multi-instance, recovery, consistency, delivery order, etc. A basic implementation is probably easy to do, but the other 80% will take the another 800% of time to implement
Everytime axon loads an aggregate it loads data and locks it.
If anything goes wrong on the command handler, nothing will be saved to the repository.
So if event publishing fails, it will fail the whole command.
So basically you do not persist anything while handling the command, you persist after command handling + event publishing.
(from my understanding)
You are right: if the event publishing fails the changes are rolled back and the system is in a consistent state.
But: What if the event is published successfully , but the command handler which emitted the event cannot persist its changes (because the database connection is broken or a database constraint is violated)? In this case, the event is published, although the command has not been processed properly.
I have to admit that I did not consider which side effects the usage of an outbox might have. What I ask myself is: Could it be worse than having an inconsistent state (I do not talk about eventual consistency which is absolutely fine)?
You mentioned a couple of caveats like increased latency, multi-instance or recovery which might apply to certain scenarios … and I’m pretty sure you are right, that the 80/20 rule fits in this case.
But: Currently in 100% of the cases Axon cannot guarantee a consistent state (concerning command handling and event publication). For sure the switch to ES is a valid recommendation, but then I wonder, if those state-stores aggregates have a future in Axon…
Please don’t get me wrong: The last thing I want is to discredit the Axon framework. It’s a great piece of software. I am an open source enthusiast and that’s the reason why I try to improve the frameworks I love by contributing code or providing (hopefully valuable) feedback / enhancement ideas.
Ah, yes, thank you Oliver for opening my eyes a bit. I think that you are right, axon cannot properly keep the consistency in state stored aggregates for state and the events without that pattern.
That was also one of the problems which I've came across at first, but i did not even bother researching transaction models, i've just jumped straight into event sourcing, it provides me one source of truth and handles consistency at the core level of my communication.
But it is quite a thing that i haven't seen mentioned in the axon documentation, there isn't much of explaining how those things work internally and how consistencies are kept, it would be a really good thing to mention in the docs.
Let me guess that there are only two solutions, eliminate state stored aggregates in favor of event sourced ones (it really feels super weird for me to do any state changes outside event handlers now ;o), try to solve such problem with that pattern.
