Axon Saga Implementation and transactions

Hi All,
I have few doubts over saga.Could you please confirm whether axon supports sagas in both ways orchestration and choreography?Or Axon supports only orchestration based sagas?

Saga definition says it is used to manage transactions across aggregates or bounded contexts which are running as distributed services.
But even in a single bounded contexts with asynchronous command bus and event bus there no guarantee that both event saving to event store and actual event handling happens in a single transaction.So how to ensure both event saving to event store and event firing and event execution is holistically successful or fail.Do we need to incorporate Sagas in this single bounded context with asynchronous buses to achieve transactionality across various levels at command bus and event bus?

Please clarify.
THanks,
Isuru

As is the case with most applications, it depends on what you’re trying to achieve which angle makes sense. The answer’s going to be rather large, but I feel it’s required to provide the necessary background. So, here goes.

The process of publishing an event and handling an event are, in most scenarios, separated from one another.

More specifically put, Axon uses the EventBus/EventStore interfaces for publication and storage. Note that from Axon’s point of view, the EventStore is nothing more than a specific implementation of the EventBus which adds the option to append events to the store.

When it comes to handling events, Axon uses so-called Event Processors. The EventProcessor has a couple of implementations, which differ with how they receive their events. Firstly, there’s the SubscribingEventProcessor.

The SubscribingEventProcessor (SEP for short) uses a SubscribableMessageSource to receive its events from. It is the EventBus that implements the SubscribableMessageSource, which makes it the default source for a SEP’s events. Furthermore, these events are provided in the same transaction as they’re published. That means that if event handling supported through a SEP fails, the event publication would essentially be rolled back. Simply put, the SEP uses an event-push mechanism.

The other type of Event Processor is the so-called StreamingEventProcessor. At this time, the StreamingEventProcessor has two implementations, (1) the TrackingEventProcessor (TEP) and (2) the PooledStreamingEventProcessor (PSEP). The StreamingEventProcessor implementations use a different message source since they’re required to stream the events. This is the so-called StreamableMessageSource. Furthermore, it is the EventStore which is an implementation of the StreamableMessageSource, as it can provide an application with the capability to start at any position in the stream/store. Lastly, the StreamingEventProcessor retrieves events on its own in separate threads and thus separate transaction. Simply put, a Streaming processor uses an event-pull mechanism.

Now, with that piece of background, I feel there is some common grounds to move to the Saga.
The Saga is in essence a specialized type of an Event Handling Component. In Axon, Event Handling Components receive events by the Event Processor. So, it is the Event Processor that invokes the @EventHandler/@SagaEventHandler annotated methods that you would construct in your application using Axon.

If you would require “event saving to event store and event firing and event execution” to be part of the same transaction, your only option would be to use a SEP as the processor for your Saga. Doable, with some configuration, but in essence not the recommended usage of a Saga. Axon defaults Event Processors to the TrackingEventProcessor, as this approach separates your transactions. In essence, it encourages your application to work in an uncoupled fashion from one another, which is most definitely the strong suit of using a Message Driven Architecture as supported through the framework.

Instead of trying to make sure that event publication, storage and handling are a single transaction, we recommend that storage and handling are separated from one another. Essentially the fact the event was stored is the bread and butter of an Event Sourced system. Furthermore, if something isn’t happy with the event (e.g. event execution fails), that’s of no concern of the event publisher.

The decision leading to the fact/event has been made, whatever some other process thinks of this does not influence the original decision making process.

Now, to circle back to whether Axon supports either Orchestration or Choreography, I’d state that you can achieve both with Axon. You could have a Saga that orchestrates the entirety of your complex business transaction, or you create several Saga’s which choreograph with one another to achieve the same goal. As far as I was concerned, the specifics about transactions spanning event publication/handling didn’t play into this part to be honest, although I might have misunderstood things.

What using a Saga would mean in your application, is that you have a component that reacts to facts (events) and tries to perform an action (command, 3rd party operation) based on it. If the action fails, it is the Saga that needs to deal with this by performing a compensating action, if necessary.

Hoping this sheds some light on the Saga’s place inside an Axon application, @isuru.

So its the command bus which passes messages to aggregates to handle after validation and then message is passed to event bus to store it in event store and handle the query possibly to update query model.

As you rightly said SEP will process events and ensure event storage is successful in same tx if its simple command bus.How about when asynchronous command bus is used?How to assure the atomocity of command handler and event handler?

And when streamingeventprocessor is used its completely a different thread there also how to ensure atomocity of command handling and event handling?

So you are suggesting to use TrackingEventProcessor where event handling happens in a totally different thread.How to notify the caller about event is at least stored in event store because it is the responsibility of event bus.And commandhandler will return once message is passed to event bus when there is no sufficient info about event is stored or not in eventstore?How to tackle this?

It seems Saga transactions only wraps “series of service calls corresponding to series o events after a command”.Saga has no responsibility of ensuring transaction from command bus/command handler to the completion of all the events.Simply Saga tx wraps around the series of events after command handler pass the message to event bus?Please clarify.

Thanks,
Isuru

How about when asynchronous command bus is used? How to assure the atomocity of command handler and event handler?

The asynchronicity of the command bus does not play a role in whether command handling + event handling is atomic. An async command bus simply embraces the fact the command is sent over the wire and the result will come in later. Dealing with that result can be done with a CommandCallback or through the returned CompletableFuture of the CommandGateway.

And when streamingeventprocessor is used its completely a different thread there also how to ensure atomocity of command handling and event handling?

Short answer: you cannot. You have distinct thread pools doing the work and the transactional solution in Axon does not employee distributed transactions (with good reason).

To have command handling and event handling occur in the same transaction, you simply use the SubscribingEventProcessor, which I’d like to rehash is not recommended for Saga’s in general. Embracing the asynchronous nature is what provides benefit in CQRS/DDD/ES systems, as it gives the highest level of flexibility. Dealing with exceptional results as they come back in the callback/future is what’s recommended, typically by performing a compensating action.

And commandhandler will return once message is passed to event bus when there is no sufficient info about event is stored or not in eventstore?How to tackle this?

The Command Handler returns as soon as the process of handling is done. The process of handling includes:

  • Intercepting the command prior to handling
  • Invoking the command handler
  • Invoking any event sourcing handlers which follow from an event publication
  • Storing the event to the EventStore

Thus if command handling fails, you are certain the event isn’t stored.

Simply Saga tx wraps around the series of events after command handler pass the message to event bus?

The transactions within a Saga are per batch of events the Event Processor is handling. By default, that’s a single event. Axon starts what’s a called a UnitOfWork to encapsulate any message within an Axon application. You can assume that any handling method will have an active UnitOfWork, by the way.

Thanks steven for detail clarifications.

Earlier reply you mentioned " More specifically put, Axon uses the EventBus /EventStore interfaces for publication and storage. Note that from Axon’s point of view, the EventStore is nothing more than a specific implementation of the EventBus which adds the option to append events to the store."

This reply implied its the eventbus which does append the event to event store.Thats why it was misleading.You replied correctly in the most recent reply.

What are the possible options for custom event buses?Can we have Kafka or rabitmq as event buses and still have Saga functionality?OR we have to rely on the eventbus in the axon server node?
Can we use KAfka RabitMq etc for command bus as well?
Please clarify

Thanks,
Isuru

I get the confusion around the bus/store, but I do feel the wording follows the correct pattern.
An EventStore is an implementation of the EventBus.
Thus, whenever you publish an event, which in Axon is always against the EventBus, it is the implementation of the EventBus which dictates whether it is going to be stored as well.
As in most scenarios the used EventBus is an EventStore implementation, you’d thus not only publish but also store the event. Per reference, here is the EventBus interface and here is the EventStore interface.

What are the possible options for custom event buses?

I feel this question is completely out of the scope of the original question, to be honest.
For clarity, to any other readers of the forum, it would be best to move this to a separate question or to search the response (as I am 100% confident this has been answered more often).
Nonetheless, that wouldn’t be extremely helpful, so please regard this is a future consideration whenever you have another question.

A “source of events” within an Axon application can be, as stated earlier, two things: the SubscribableMessageSource and the StreamableMessageSource. As those are interfaces, you can essentially construct any source you’d want. What’s currently supported, however, are the following:

  • Axon Server Standard → subscribable (for local) and streamable source
  • Axon Server Enterprise → subscribable (for local) and streamable source
  • An RDBMS with shared data source → subscribable (for local) and streamable source
  • AMQP → subscribable source
  • Kafka → subscribable and streamable source

If you are venturing on “what the best solution would be”, I’d like to assert you that there is a reason why we’ve position Axon Server in this place. Taking a look at this video might clarify this intent in a bit more detail.

Thanks Steven.I’ll take up additional questions in separate threads as you suggested.
Appreciate if you can pay attention to this new topic since no one has replied yet for 3-4 days.

Regards
Isuru