Hi,
I’m implementing Axon framework for a web app using a NoSQL database: DynamoDB.
Do you plan to implement an event store for DynamoDB ?
Thanks in advance.
Nicolas GUERRIER
Hi,
I implemented a dynamo event store. Unfortunately, ithere are still many problems I have to solve. The problems are mostly edge cases but there is no margin for error iwhen dealing with such important data.
You will encounter certain difficulties related to the hard limits of dynamo. For example, an event cannot be larger than the maximum size of an item, which is 64 kb if my memory serves me right. Another limitation is that inserting multiple items at once is not atomic. True atomicity can only be accomplished when operations are limited to one item. You could in theory end up with an inconsistent state if there is some level of failure, either in your application or in dynamo. Using one item per tx (like one of the storage strategies of the mongo event store) is almost impossible, considering the size limitation of an individual item.
But to be honest, these problems are trivial compared to the synchronization problems you will encounter. Since dynamo has no locking mechanism, you will need to implement one yourself. I did this using a separate table with one lock per aggregate. The item also contains a reference to the last consistent sequence number; the aggregate can thus still be read in a consistent state, but writing is impossible because the lock will not be available. This approach also has some serious problems. If for whatever reason, a lock is not released once the events are succesfully appended, the aggregate will be locked forever or untill someone manually releases it. I have thought about many possible solutions to this problem but still have not found a satisfactory one.
It is also very important that ALL reads are done with eventual consistency disabled. This affects performance but it is the only way to make sure you are getting the real event stream.
Given dynamo’s design, i have serious reservations on its usefulness as an event store.
Hi Nicolas,
I haven’t looked at DynamoDB yet, so that’s probably your answer. ;-).
For now, Dynamo isn’t on the roadmap yet. But I haven’t rejected it as a viable solution either.
Cheers,
Allard
Hi Nicolas,
Thank you very much for your clearly response. After reading your comment I do think also that DynamodDB is unfortunately not suitable. I had not thought about the consistency read and many points that you explain. Is that the problem of atomicity can be solved thanks to the transaction libray for DynamoDB (http://aws.typepad.com/aws/2013/07/dynamodb-transaction-library.html)? Do you think DynamoDB can only be used side query?
Nicolas GUERRIER
Hi Allard,
Thank you for your response but after reading Nicolas A. Bérard-Nault I think seriously that an event store with DynamoDB is compromise.
What do you think about this ?
Hi,
Nicolas’ (the other one 
) analysis sounds very reasonable. I have come across other (NoSQL) database implementations that have shown not to be suitable because of their design principles. Based on his email, I would also assume that the choices made in Dynamo simply don’t match the requirements you’d typically have in an Event Store.
Since you’re on Amazon, you could consider using RDS for storing events, using either the JPA event store or the (new) JDBC one.
Cheers,
Allard