Publishing to Amazon SNS with at-least-once delivery

Imported from Google Groups
1

I’ve read that Axon does not support Amazon SNS out of the box. What would be the easiest way to publish events to SNS with at-least-once delivery? My first thought was to simply wire up a global event handler (i.e. a method with @EventHandler and a GenericDomainEventMessage<?> parameter) and have it publish to SNS, but then I’d have to do all the error handling myself, especially with publishing events that were persisted to the event store but failed to publish. I know that Axon supports publishing events to AMQP and Kafka. Is there a way to write a custom event publisher that uses the same interfaces that these use? Does that interface guarantee at-least-once delivery?

2

Hi,

if you look at how the RabbitMQPublisher and KafkaPublisher are built, you should have a good foundation for building one based on SNS. This will also guarantee that events are published to SNS, before any local transactions are committed, if SNS supports transactions in some way. Check out how the RabbitMQPublisher hooks into the Unit of Work to synchronize the publisher transaction.

If SNS does not support transactions (I am not intimately familiar with the API), you’d have to publish it there in the “after commit” phase of the unit of work. Not doing so could result in events being published that you don’t have in your event store, which is a bad position to be in. But now, you have opened up the other way round: events are stored, but not published. A workaround could be to build this into a Tracking Processor, which by design only reads events available in the event store. In that case, you can easily guarantee at-least-once-delivery.

Hope this helps.

Allard

3

SNS does not support distributed transactions. Agreed that an event would have to be published to SNS after the unit of work commits the event to the event store. Would using a tracking processor that persists its token to a JDBC token store ensure that each event gets published at least once? If an app crashes after the unit of work commits an event but before a tracking processor is run for the event, then after restarting, will the app use the persisted token to pick up where it left off so that each tracker is given each event at least once?

4

I played around with the tracking event processors and I have a couple of follow-up questions:

  1. What is the preferred way to set up tracking event processors? The documentation says to use EventHandlingConfiguration and call usingTrackingProcessors() but this method is deprecated. Is there an example somewhere for how to set up event tracking processors using a persisted (e.g. JDBC) token store in a non-Spring app?
  2. Is there a way to prevent a tracking token from incrementing if the event handler throws an exception? I tried setting the EventProcessingConfiguration.configureRollbackConfiguration() to ANY_THROWABLE, but the tracking token was still incrementing even when the event handler throw an exception. For the SNS publisher, if it fails and throws an exception, it never gets retried, so we can’t guarantee at-least-once event publishing.
5

Hi,

  1. While that method still works, we recently introduced the EventProcessingConfiguration class. That’s the new place to configure processors.

  2. If you want to retry processing after event handler failure, configure an EventHandlerInvocationErrorHandler that propagates the exception to the tracking processor. That will cause events to be retried, unless the processor has an error handler that does otherwise.

Hope this helps.
Cheers,

Allard