I have a standard Spring-Axon app (that mostly uses axon autoconfiguration) that uses a distributed command bus (jgroups) that I’m running on 3 nodes. The command side seems to be working nicely but I have some questions regarding event handling and tracking processors in multi-node configuration.
Do I have to do anything specific in Axon 3.0.x to not run into problems on the event handling side when running multi-node configuration? I’m running into a weird issue where after some time my tracking processors on different nodes all start to process the same event and I can’t seem to figure out what is the problem. After restarting the app everything goes back to normal for a while- only one node processes the event. I’m not sure what can I do to debug this. I use JdbcTokenStore that uses ManagementFactory.getRuntimeMXBean().getName() as nodeId.
I’m having problems with understanding how to do a proper transaction management configuration when using a token store and tracking processors . Let’s say I have an eventstore in database 1 and a single projection in database 2. Should the token store be kept in database 1 or database 2? The tokenstore mechanism uses a TransactionManager. Is it fine to have the same TransactionManager handle the eventstore and the projection if they are kept in separate databases?
usually, for Event handling, you don’t need to configure anything “special”. By default, handlers are executed in the thread that publishes events, which basically means ordering of events is based on the aggregate that published them. If your handlers expect different ordering guarantees, it is better to handle these events asynchronously (e.g. using a tracking processor), so that publication and handling are done in different threads.
Regarding 1. Tracking Processors (by default) will update the token each time an event was processed (or a batch, if configured). While no events are available, they will update the timestamp of the token only, once each second. A tracking processor that is not handling events, will attempt to “steal” a claim by another, and will only succeed in doing so when the claim hasn’t been extended for (by default) 10 seconds. If multiple instances of a tracking processor are handling events, this may mean that they aren’t reading from the same TokenStore (i.e. database) instance.
Regarding 2. Ideally, the TokenStore uses the same database that also stores the projection that the events update. This way, a single transaction will commit both the token, and the changes to the projection. Reading from the Event Store may happen in a different transaction, if any at all.