I am starting a project based on event-sourcing and we decided to adopt Axon.
We have the following ideas:
Store the event using AWS Dynamo
Propagate the event using AWS SQS
AFAIK this not supported by default in AXON, so I was thinking about using Axon just as a CommandBus/EventBus framework. I mean, once a command is executed then the events are created and propagated, later we should find our own way to read the events from dynamo and send them over SQS to rebuild the application state.
I would like to hear if this approach is not a good idea or if this feasible or any other opinion.
I am not intimately aware of the consistency features of Dynamo, but I assume they are similar to Cassandra. We have found that Cassandra is not suitable as an Event Store. My colleague Frans has given a webinar on this matter, where he outlines the features required, and how (No)SQL solution do or do not meet these requirements: https://www.youtube.com/watch?v=H20hh5oXzrk
Building a Connector to publish (and read) from Amazon SQS shouldn’t be hard. You can use the AMQP as examples. But in our experience, it a lot simpler of Event Handlers read their events directly from the event store, instead of using another source of messages. Separating into two sources makes your infrastructure more complex and has a negative impact on throughput.
Hope this helps you a bit.
Cheers,
Allard
Note that you can always get in touch with us at AxonIQ, in case you need more detailed / tailored advices based on what you’re trying to achieve.
Great to hear that you’re considering Axon. You’re right, AWS Dynamo/SQS is at the moment not supported directly by Axon at this point - some more technical details on that later.
First of all, I’d like to comment on more fundamental issue: you’re talking about reading events “to rebuild the application state”. That sounds like you’re planning on using event sourcing as pattern. Axon Framework is very well suited for that. You would see state being event sourced in two ways:
The actual aggregates processing commands would only store their state as a series of events. This is what I would consider “true” event sourcing because events would be primary storage. This way of event sourcing is not mandated by Axon, but it is in a certain sense the default and it is used by a large majority of users.
Read-side/query models can be populated using events raised by aggregates in command handling or somewhere else.
Now, “using Axon just as a CommandBus/EventBus framework”, ignoring the storage of events momentarily, will work if you’re satisfied with doing (2) without doing (1). If you want to do (1), you’ll need a way to store and send events from the beginning. I’d definitely recommend doing (1) and (2) together. Ultimately, it’s the easiest approach. If you’re doing (1), then by definition you will have events covering everything that happens and (2) will be easy.
To run Axon (both (1) and (2)) on Dynamo/SQS could potentially be achieved by creating custom implementations of the EventStore, CommandBus and QueryBus interfaces, but you would face quite a few challenges. To mention just one: SQS standard queus don’t offer an ordering guarantee, which is a very bad match with event sourcing! (Imaging a ‘OrderCreatedEvent’ and ‘ItemAddedToOrderEvent’ being delivered out-of-order). Now, SQS does alternatively offer FIFO queues, but they’re not available in all regions and have throughput limitations.
I think you have some options that would make for an easier, more scalable setup on AWS:
You could use AxonHub and AxonDB to get a very easy to maintain, highly scalable Axon infrastructure. This could run on Amazon EC2 instances, but also works great with Kubernetes, which will soon be supported natively on AWS (currently available in preview). Kubernetes in general is a great match with a scalable Axon application.
Using an AWS RDS database as an EventStore will work as well. Axon’s tracking event processors would be actively reading events from there by doing polling for new events.
Hope this is useful - happy to discuss this further in a call or something if you want.