TokenStore/SagaStore + multiple application nodes

Imported from Google Groups
#1

Hi there,

I have a question regarding running multiple nodes of the same application connected to the same axon-server (for scaling).

I try using JPATokenStore and JPASagaStore for persisting information. Does these stores need to be actually in a shared database for all application instances (nodes) or does every application instance (node) have its own database? Since i encounter sagas being replayed even tough they should not when replaying events (according to this answer: https://groups.google.com/d/msg/axonframework/_v2yuIL-lwQ/_GdC1enKBgAJ) i assume that I do something wrong.

I would be grateful if you could give me a hint on the right application layout or point me to the right place in the reference documentation.

Thanks and Best Regards,
Jakob

#2

Hi Jakob,

Firstly, the database-sharing:
I’d assume that the token and saga tables for one ‘type of application’ would be shared among all the instantiations of that application.
That’s at least how I have been used to setting this up.
I however wouldn’t reuse the token/saga tables between different types of Axon application among your application suite.

Now for the ‘saga replay’ point:
The replay of a saga is definitely blocked, in code.
However, if it’s a Saga which has never run before, whilst a suite of events already exists on the application it’s added to, then it will initialize itself by traversing the entire event stream.

If you do not want your new Saga implementations to start reading events from scratch once they’re introduced, you should adjust the TrackingToken of the Tracking Event Processor which backs up your Saga.

This is achievable if you specify a TrackingEventProcessorConfiguration which initializes the Tracking Token to the head of the stream (though the static andInitialTrackingToken() function on the TrackingEventProcessorConfiguration class).

This TrackingEventProcessorConfiguration object should then be provided when your register your Tracking Event Processor, in turn doable by using the EventProcessingConfigurer#registerTrackingEventProcessor() function.

I hope this helps you out Jakob!
If not, feel free to reach out again.

Cheers,
Steven

#3

Hi Steven,

thanks for your clarification. I guessed right and already restructured our application the way you suggested (shared database for multiple instances of the same application).

The encountered Sagas replaying behaviour was due to each instance of the same application having their own database: when I was spinning up the first instance of the application with an empty local database it applied all events from Axon Server until the head of the Event Stream and executed also the sagas (since it was initialization). Stopping and restarting this application instance was then fine, because the Token Store was persistent and the application knew that the events until the Token location were a replay so no Sagas got triggered. The problem started when spinning up a second instance of the same application with their own, now empty database. This instance also applied all events from Axon Server until the head of the Event Stream because it had no Token location in the database, so also all Sagas got triggered again on this application instance which led to duplicate events/commands executed.

Now having a shared database for those multiple instances of the same application everything works as expected.

A side note: thanks to the Axon team for providing such a well designed and maintained framework. Once i took the initial hurdle of understanding the concepts behind CQRS and Event Sourcing :slight_smile: I experienced a steep learning curve applying those concepts in Axon due to the straight APIs and documentation. Keep up the good work.