Distributed EventBus

Imported from Google Groups
1

Hi all

we are currently setting up a distributed event bus, using cloudamqp(rabbitmq) and the basics seem to work fine.
But we have multiple apps that should retrieve the events, and i was wondering how to make sure that all listeners receive the message.

If one component is down for maintenance, or unexpected causes we want it to continue processing where it left off, instead of having missed a couple of events. What would be the best way to do this?

I could come up with having a queue per component, or using a kafka like system in which the client saves the state where it left off, but i’m no expert on amqp, and don’t know if there is a normal pattern to solve this problem.

So if someone has a tip, please let me know.

Cheers,

Job

2

Hi,

on AMQP, you should have a queue per application. When multiple instances should compete for events, they must read from the same queue. Otherwise, each should have its own.
Kafka is also a good option. Basically, each consumer (application) will track the stream, keeping its own offset.

Cheers,

Allard