Multiple Axon instances working on the same event store

Imported from Google Groups
#1

Hi :slight_smile:
We have an application using Axon on an event sourcing repository. Everything is running fine but we are wondering: what happens if more than one instance of the application use the same event store? Let’s say two completely separate (but otherwise identical) java processes that share the same physical database server. Would any of the axon instances encounter errors or weird behavior due to the shared database or does axon have some kind of consistency check that would prevent concurrency problems when different users on different instance work on the same aggregates?
Thank you for your time.

Giacomo

#2

You’re should use the distributed command bus. Then you can run multiple copies of the application in different JVMs. You will need to use the JDBC, JPA or MongoDb event store. The distributed command bus takes care of routing commands to a particular aggregate (by aggregate id) to the same JVM instance - this gives you the benefit of caching.

If multiple nodes do try to commit to the same aggregate from different nodes, then you’re going to get a primary key violation since the commit sequence number would be the same.

#3

That’s exactly what we were looking for. Thank you :slight_smile:

Giacomo

#4

There is something not very clear: the command is recieved on JVM1 and is possibly routed to JVM2.
This sounds good from the perspective of the domain layer (commands, aggregates, events), but may be a problem in the perspective of the application layer.

For example if the transaction is provided from application layer how can the transaction be routed to another JVM?
I can even have other things, like identification of the tenant or infrastructure services I can’t route to another JVM.

In our software the tenantId is resolved at the appliation layer and injected in the transaction so the domain has no knowledge of the tenant concept, each tenant is kept isolated at the JPA level.
I can’t imagine how this could work if the command is routed to another JVM…

Are there other strategies to allow N application instances to work on the same event store concurrently?

#5

Having multiple instances work with the same event store isn’t any problem. It’s important to make sure each aggregate instance lives in only one place. If they are used concurrently in multiple nodes, you can get the ConcurrencyException, which effectively means that a command has not been executed against the last version of an aggregate. Most likely, your MetaData Routing strategy does not route consistently for a given aggregate. You’ll either need to make sure that a single aggregate is always routed to the same node, or that you perform a retry on a Concurrency Exception.

Regarding the transactions, in message driven applications (such as an Axon based one), it is important to see a single message processing as a single transaction. Thus, processing a command is a transaction. Processing an event is another (although it is sometimes possible to do it in 1). Therefore, you don’t “route” a transaction to another JVM, but you dispatch a command (which is handled atomically) and optionally react on its result. Your code should not rely on that command being executed in the same transaction, or even on the location where the processing happens.

You can attach the tenantID as meta data to the messages. That way, your infrastructure code will know about tenant ID, but the domain logic does not. At least, you’ll be able to transport your messages across JVM’s and still know for which tenant you’re executing code. You can use CommandHandlerInterceptors to set the necessary ThreadLocal data for the tenant configuration for JPA.

Cheers,

Allard