Race Condition between Aggregate, Command, Event, and Saga

Greetings,

It seems that similar questions have been asked, but I am struggling to establish a reference to my particular example.

Here is what I have.

Aggregate - EntityAggregate
Command - CreateEntityCommand
Event - NewEntityCreatedEvent
Saga - FulfillmentProcessSaga

I believe I am using Tracking Event Handling and an AsynchronousCommandBus.

I send a CreateEntityCommand. EntityAggregate picks it up. A Transaction is started before the command handler is executed. I create a new Entity and send a NewEntityCreatedEvent event using the apply() command. The transaction commits.

My FulfillmentProcessSaga is waiting for the NewEntityCreatedEvent events to do some work. I see that in some cases the event handler on the Saga fires before the command transaction is committed and the object is actually stored in the DB. The saga’s logic fails as the object isn’t yet available.

My question is what I am doing wrong and how I should queue these events perhaps and have them sent after the transaction successfully commits.

Thank you!
Oleg

Hi Oleg,

Not quite sure what’s going on yet, so I’m firing a couple of questions at you to figure out what state your application is in. :slight_smile:

First of, am I correct to assume your FulfillmentProcessSaga queries a view which is updated on the exact same event it is triggered on itself?

Secondly, what is ‘the object’ you’re talking about in your request? Is that a view? Is that view instantiated by the NewEntityCreatedEvent?

The Saga Event Handler being called prior to the command transaction being committed to me sounds like the Event Processor for you Saga is ran in the same thread. In turn, that suggest you’re not backing it with a Tracking but a Subscribing Event Processor.

Lastly, you’d shouldn’t be bothered with queueing your events prior to the command handling transaction commits successfully.

Hope we can figure this out!

Cheers,

Steven

Hi Steven

Thank you for replying! Here are the answers:

  1. No, the Saga in question queries data that was created by a command handled by an Aggregate that triggered the event.

Aggregate Creates Data -> Publishes Event -> Saga Gets the Event -> Queries Data

  1. Well, as above, the data is created in the DB by the command on an Aggregate, and then the NewEntityCreatedEvent is published.

  2. My Saga is backed by a tracking Event processor and the event handler fires on a separate thread. I think it would work better is I had this all run on the same thread.

I did more digging. Here is what I discovered.

It seems that the events publishing happens as part of the transaction commit and I understand the logic behind it. It appears that in the commit code the events are published first and immediately after that my transaction manager’s (registered with Axon) commit method is called. This is where I think the race condition occurs.

When the event is published, my commit code hasn’t yet been called. My transaction commit followed immediately. My commit actually commits the underlying DB transaction, which created the new Entity. Since the event is published before my DB commit and event processing is Tracking and Asynchronous, sometimes, the Saga’s event handler fires before the DB commit is completed. The new Entity isn’t yet available to be processed.

My solution for now is to hold certain events in my own thread-bound context and publish them a MessageMonitor I registered with the CommandBus. In the MonitorCallback’s reportSuccess() method I get all the events queued and publish them. It appears that I can either publish before the commit or after the command is completed, i.e. the underlying UnitOfWork is fully done. Not sure if this is correct, but I am not able to find another option.

I really want to make this working right, not too comfortable with workarounds.

Thank you!
Oleg

Hi Oleg,

I get the feeling your mixing a couple of concepts, although I’m not a 100% sure about that.

So I’ll fire off with my answer to the above, but please point me in the right direction if I’m making wrong assumptions here.

From your response I get the feeling that you’re storing data in a database in a command handling function on an aggregate.

If that’s the case, I highly suggest not doing that, as it’s mixing the Command and the Query side of your application
A command handler should perform business logic based on state your aggregate contains and make the decision whether an event should happen yes/no.
It is thus not in charge of updating views, it just makes a decision of a certain action is valid or not. The event in that sense notifies everybody that that decision has been made.

After that, an event handling component in the query side of your application will handle that event.

In most cases such an event handling component will create/update a projections/view, but it might also perform certain other actions like sending an email or in the case of sagas publish another command.

Lastly, I’d suggest no to use the MessageMonitor set up for anything else than monitoring purposes, as that’s what it is intended for.

Hope this helps!

Cheers,

Steven

Hi Steven,

Thank you for your reply and the clarification! This helps a lot.

Yes, I do create/update persistent data in my command handlers. The reason I do is that the commands are published by UI controllers based on the user input to perform actions. The controllers wait for the command outcome to provide response to the UI with the newly created/updated data.

If I change this and use commands only to make a decision to create/update and perform the actual action in the event handler, how can I reconcile this with my request/response UI architecture?

Thank you!
Oleg

Hi Oleg,

What I typically like best is that the front-end subscribes to updates instead of waiting for that update to return after it’s publishes the command.

This resonates well with the fact that a command as the request to perform some action, as it is most often described in CQRS literature.

That thus suggest some component (your front-end in this case) wants to do something (thus publishes a command) and it doesn’t really care who’s in charge of performing that action as long as it’s performed.

Not actively waiting for the response, but expecting it later on through some channel you’ve described to works well with that scenario.

WebSockets, together with the STOMP protocol might be an option for that example.

Or you could return a dummy response from the service which publishes the command to have a quick reply and let it pull the actual data based on some form of notification process you could add.

You could also decide against doing CQRS of course, which would I guess eventually lead to having a completely synchronize set up from publishing your command to all your event handlers being finished handling the subsequently following events.

I guess however that’s not desirable for you situation :slight_smile:

That’s my two cents.

Cheers,

Steven

I often have to deal with similar situation (but please note I’m on 2.6) and I often find out the reason is to have a “traditional” design - sequential and synchronous - that is not very well suited to DDD modeling. In your case instead of

Aggregate - EntityAggregate
Command - CreateEntityCommand
Event - NewEntityCreatedEvent
Saga - FulfillmentProcessSaga

Aggregate Creates Data → Publishes Event → Saga Gets the Event → Queries Data

What I would do is
Aggregate - EntityAggregate
Command - StartCreateEntityCommand, FinishCreateEntityCommand
Event - NewEntityCreationStartedEvent, NewEntityCreatioFinishedEvent
Saga - FulfillmentProcessSaga

Aggregate receives StartCreateEntityCommand (updates it’s internal state only) → Publishes NewEntityCreationStartedEvent → Saga Gets the NewEntityCreationStartedEvent → Saga Creates Data → Sends FinishCreateEntityCommand

Aggregate receives FinishCreateEntityCommand (updates it’s internal state only) → Publishes NewEntityCreatioFinishedEvent-> Saga Gets the NewEntityCreatioFinishedEvent-> Saga Queries Data

I stress what Steven said that is in fact very important:

From your response I get the feeling that you’re storing data in a database in a command handling function on an aggregate. If that’s the case, I highly suggest not doing that, as it’s mixing the Command and the Query side of your application

A command handler should perform business logic based on state your aggregate contains and make the decision whether an event should happen yes/no.
It is thus not in charge of updating views, it just makes a decision of a certain action is valid or not. The event in that sense notifies everybody that that decision has been made.
After that, an event handling component in the query side of your application will handle that event.
In most cases such an event handling component will create/update a projections/view, but it might also perform certain other actions like sending an email or in the case of sagas publish another command.

And I actually would go a step further to say that Aggregates, besides raising events based on the business decisions, should do nothing except taking care of it’s internal state - specially they should do nothing that affects “the external world”. All those things - writing to models, iterating with other systems, etc… - should be done either by simple listeners or sagas if they are complex, long lived transaction that need to keep their own internal state, and that in turn should send commands to notify the aggregate if needed.

Much more verbose but that’s the price to pay to have asynchronous processing and all the DDD/Event driven/CQRS styles of processing.