Confised about AggregateIdentifier uniqueness...

Imported from Google Groups
1

Hi,

I’m having a bit of trouble understanding the behaviour of AggregateIdentifier when used with MongoDB and SpringBoot.

I seem to be able to easily replay ‘Create’ commands using the same AggregateIdentifier (a UUID or even just “1234”).

I’ve exposed a rest endpoint which takes ‘create’ commands and turns then into Java commands with @TargetAggregateIdentifier annotations that can be processed by Axon. However, if I repeat the request with the same aggregate ID many times, rather than getting some kind of exception like MongoException.DuplicateKey Axon just seems to happily continue creating duplicate Aggregates.

Here’s some output resulting from 2 calls from Mongo:

{ “_id” : ObjectId(“56a1454a3f96ab79b9a78e1f”), “aggregateIdentifier” : “1234”, “sequenceNumber” : NumberLong(0), “serializedPayload” : “<com.soagrowers.product.events.ProductAddedEvent>1234Awesome Drill</com.soagrowers.product.events.ProductAddedEvent>”, “timeStamp” : “2016-01-21T20:53:30.921Z”, “type” : “ProductAggregate”, “payloadType” : “com.soagrowers.product.events.ProductAddedEvent”, “payloadRevision” : null, “serializedMetaData” : “”, “eventIdentifier” : “28c189d5-def4-4400-8ae0-aa995841f1ff” }
{ “_id” : ObjectId(“56a1454b3f96ab79b9a78e20”), “aggregateIdentifier” : “1234”, “sequenceNumber” : NumberLong(0), “serializedPayload” : “<com.soagrowers.product.events.ProductAddedEvent>1234Awesome Drill</com.soagrowers.product.events.ProductAddedEvent>”, “timeStamp” : “2016-01-21T20:53:31.845Z”, “type” : “ProductAggregate”, “payloadType” : “com.soagrowers.product.events.ProductAddedEvent”, “payloadRevision” : null, “serializedMetaData” : “”, “eventIdentifier” : “793cc97b-1803-44fb-a7ff-5217a8c344aa” }

As you can see, the AggregateID’s are identical. However, the ObjectId and eventIdentifiers are both different. I’m not sure how these get created, but I realise they may have an impact on uniqueness.

Where am I going wrong? Can anyone help?

I’m using Axon 2.4.1 with the ‘mongo latest’ docker image on ubuntu 14.04.

My full code is here: https://github.com/benwilcock/microservice-sampler

Cheers

Ben

2

Hi Ben,

the ObjectId is an identifier created by mongo itself. The eventIdentifier in this case just shows that the two events with the same aggregate identifier and sequence number are in fact from different events (even though they have the same content).
Did you create a unique key on type, aggregateIdentifier and sequenceNumber? That should prevent double values to be inserted. It is usually automatically created, as the method to invoke is annotated with @PostConstruct. However, if you create and manage the instance manually, you may need to invoke ensureIndexes when creating the MongoEventStore instance.

Cheers,

Allard

3

Hi Allard,

Thanks for responding. I did wonder if the ‘ensureIndexes’ was being run, so I added it as a specific task for Spring to perform during the startup of my app. However, what I found was that although the ensureIndexes was good at telling me I had a duplicate entry in an existing database, but it didn’t seem to prevent the database entries being duplicated in the first place. Having the index didn’t seem to prevent duplicates from being written.

I’ve since also tried stepping through the axon code to familiarise myself with all the actions being performed by the MongoEventStore and supporting classes. However, at the point where I’d expect the DuplicateKey exception to be thrown, none was forthcoming. I think it’s going to take more time for me to get to the bottom of it. I’m sure I’ve just missed something, but I’m not sure what. My goal is simply to make sure that AggregateRoots can only get constructed once and that the database only contains a single construction event for any given aggregate.

If seeing my code helps, you can find it at http://github.com/benwilcock/microservice-sampler

Cheers

Ben

4

Hi Ben,

ensureIndexes attempts to create the indexes, if they don’t already exist. However, if there are already two “competing” values, the index cannot be created, and ensureIndexes fails. You must solve the duplication and call ensureIndexes again.

Usually, ensureIndexes is called against an empty event store.

Cheers,

Allard

5

Thanks Allard,

I think that’s what I did - I ran ensureIndexes against an empty event store, but I will check that and come back to you with any findings.

Just so I’m clear, if I start with an empty store, apply ensureIndexes at boot time and then run a test that attempts to send the same ‘create’ command to the command gateway twice, I should expect to see a DuplicateKey exception and see only one event record in the event store? Is that right or is there more work I should be doing to ‘solve the duplication issues’?

Thanks for the guidance.

Cheers

Ben

6

Hi All,

I did some integration testing with Axon 2.4.3, Rabbit & Mongo and as far as I can tell some strange things are happening inside Axon that could result in data inconsistencies in materialised views.

I set up a test that would send duplicate ‘constructor’ commands into Axon’s command gateway to see what would happen. As part of the setup of the test I make sure that ‘ensureIndexes’ has been applied to a clean mongo data store at boot time as I understand that this helps protect the event store from duplicates. In a way it does, but I did observe some strange results that I wasn’t expecting, as follows…

Commands Sent: 2 (both are 'constructor commands for the same aggregate ID, but each has a different aggregate ‘name’)
Event Store Entries Written: 1 (one entry for the first command)
Events applied to aggregate: 2 (this is at odds with the number of events stored)
Events Published: 2 (again this is at odds with the mumber of events stored)
Exceptions thrown: 0 (as far as I can see, no Mongo Duplicate Key exception is triggered even through it appears the ‘ensureIndex’ is working correctly)

While I’m pleased that the actual Event Store seems consistent with what I’d expect, to me (a noob) these test results are a bit troubling. I’m not sure I want constructors to be called with duplicates, and I’m not sure I want unpersisted events to get published out to event handlers as it could result in them having an inconsistent or inaccurate view of the true domain state.

My test code is here: https://github.com/benwilcock/microservice-sampler/blob/master/command-side/src/integration-test/java/com/soagrowers/product/ProductCommandApiTest.java - see ‘testDuplicateAddCommandType2’ for details of the procedure.

Is this a bug or have I missed something?

If this is expected behaviour, would someone with a better handle on Axon please be kind enough to point out to me where I’m going wrong with my analysis or understanding? If I’ve made a mistake with my configuration where or how do I fix it?

Many thanks

Ben

7

Which Mongo version are you using?
I vaguely remember that there was a specific Mongo version where the exceptions weren’t coming accross as expected. I have tested against 2.4.8, and it behaves like expected.

Cheers,

Allard