Axon, JPAEventStore MySQL and partitioning

Imported from Google Groups
1

Hi,

We’re currently looking into using MySQL partitioning with our event store to be able to more easily manage the ever growing amounts of data. The change we’re looking to introduce is to add a new artificial primary key consisting of an auto_increment and a timestamp (MySQL timestamp instead of the Axon default varchar) and then do range partitioning on the timestamp.

One caveat is that when you employ partitioning, all unique indexes have to contain the partitioning key which means the natural key for domain events, (type, aggregateIdentifier, sequenceNumber) can no longer be created as a unique index. Given that all inserts for a particular aggregate type only happens from a single JVM and using an EventSourcingRepository (which is a LockingRepository), what problems do you see with scrapping the unique constraint on the natural key index?

2

Hi Sebastian,

when you use a pessimistic locking mechanism on your repository (which is default) and you use only a single repository instance, you should never have any duplicate inserts for the type, aggregateIdentifier, sequenceNumber combination.
Something you might have already considered, but why not partitioning on the aggregateIdentifier field?

Cheers,

Allard

3

Hi Allard,

And thanks for confirming.

The main problem of the ever growing event store would not be solved if we partitioned only on aggregateIdentifier. Right now we are spending considerable amount of IO to delete old entries after they are archived, and we’re hardly able to prune as fast as we’re appending during some periods. A drop partition would consume close to no IO and would be instant in comparison.