Replay of events in distributed micro services

Imported from Google Groups
#1

I am implementing ES / CQRS ausing axon 3.3.
In the command part, I received comands that aggregate apply and store corresponding events in a event store using MongoDB as storage.
In the query part, I have a event handler that update a separate database to build a optimized read model. Great feature is that I can replay events using tracking event processor. It works fine, at least for the moment as my commands and queries part execute in the same JVM.
Now I want to separate these two parts in two independant microservices (may be later I will have A third micro service with another database for another optimized read model). How to send events between these microservices, and how to replay events in read micro srvice in order to initialize its read model. I have used rabiitmq and amqp to send events; it works fine but does not allow to replay events.

More generally, in microservices architectures, how newly created microservices can replay past events ? AMQp does not seem an option. Should I use EventStore or Kafka for the event bus in commands part, and not process events in the queries part but rather subscribe to streams or projections ?

Thanks in advance to those who will participate in this thinking.

#2

Hello Fred,

I’m not a maintainer of the Axon framework but as explained in this article https://www.infoq.com/news/2018/07/axon-query-subscriptions-kafka, Axon 3.3 comes with a very exciting feature : the ability to replay the events from a Kafka topic. This is not possible when using AMQP as the messages are removed from the queues once consumed.

So as far as I understand, locally in your JVM, you keep using the Event Bus/Event store provided by Axon, but the event messages are then automatically forwarded to Kafka, that can be seen as a streamable message source by the other microservices. Therefore, they could trigger a replay without impacting the source module. @Axon maintainers, of course, please correct me if I’m wrong :wink: !

Btw, as the linked article mentions, you can find more details about the related PR here : https://github.com/AxonFramework/AxonFramework/pull/573/files

Best Regards,
Jerome

#3

Hi Fred, Jerome,

that’s all very correct. In Axon, the AMQP Message Source is only subscribable, as the messages disappear once read. Kafka Message Source (since 3.3) is a StreamableMessageSource, which means you can replay them.

In general, we recommend using a proper Event Store when using event sourcing. At AxonIQ, we provide AxonDB, which allows you to seamlessly integrate applications by publishing events to it. In combination with AxonHub, it becomes very easy to distribute message between nodes transparently.

Hope this helps.
Cheers,

Allard