Distributed Command bus design implementations

I have created a few services using Axon 3 that run in AWS. Up until this point there was no necessity for these services to communicate with each other. Therefore, upon receiving a request the controller endpoint sends a command, which is either handled by an aggregate command handler or by a command handler class that will load the aggregate. In either case, the aggregate will apply the corresponding event. These events will be handled in a query event handler class that will persist to the query repository. In some scenarios there is also an amqp handler that will send the event to an exchange that a legacy application (not service) is subscribing to.

I am now looking at designing a solution that would allow 2 of these services to communicate.

Scenario: Service A receives a request, sends a command that is handled internally (same JVM), and an event is applied. A subset of the payload of the request needs to be sent to service B. Service A needs to know if service B handled it’s piece or not (whatever the not reason may be… potentially unreachable) and in case of failure A sends a command to “rollback” (apply an event). In a successful request, service B will internally apply it’s own event (not sure if as a result of handling a distributed command or if it receives from A the actual event it needs to apply…).

I’ve been reading about Distributed Command Bus, but I’ve also read comments here and there on this google group about distributed Events.

  1. Should I be distributing Commands or Events?

  2. My lead architect prefers an approach that uses AMQP as the vehicle of communication, mainly to account for the fact that Service B may be unavailable. I personally don’t love this approach, because since I need a response, then Service B needs to send another message to another queue that service A needs to subscribe to. Unavailability should be VERY rare anyways, but a response is always required in case of some internal failure either from business validation or an unexpected exception happening in service B. Another thing I don’t like about this approach is, assuming B was down for a few hours and the resumed, picked up the amqp message and a failure was still generated internally and then communicated back to A for a rollback, clients accessing our application would see their changes during those 4 hours and then all of a sudden they’d be gone because they were “rolledback”.

  3. I prefer an approach where services communicate directly. If service B is unreachable, again VERY rare, then service A will “rollback” it’s internal applied event, and the client won’t even notice. In case B is up then we also get direct response if something goes wrong and “rollback” immediately, and the client won’t even notice. Even the number of characters I had to write to explain this approach are less…

  4. I have read about JGroups and Spring Cloud Connector and briefly looked at one example that seems to use amqp in some way for the communication. I am leaning towards Spring Cloud Connector (since we already have an Eureka service running in AWS), but haven’t had time to play with it and haven’t seen any demo using it.

In some cases the inter service communication will be used to provide eventual consistency of data, but in other cases it will be a way of triggering some processing that is handled by a different service.
MongoDB is used as the Event Store and the Query repo.

Really looking for some direction on what design approach to implement.

Thanks,
Bruno

No takers? :slight_smile:

HI Bruno,

The purpose of distributing commands is to scale a service horizontally: e.g. have multiple JVMs with the same service, each JVM handling a percentage of the commands (and events).
Your intent is rather to maintain consistency across different aggregates and (in worst) case roll back to a previous (consistent) state. This is exactly what Sagas are designed for.

Basically your would design a Saga (associated with the aggregate of) service A.Upon emitting event E1 containing the data needed by B, the Saga is triggered and would then communicate this information with B (eg. via REST call). If the call to B fails, the Saga must the rollback: e.g. send a command to the aggregate of service A to revert the action leading to event E1.
Keep in mind, a Saga stateful. Eaxch aggregate in service A will have it’s own dedicated Saga instance.

This resembles your approach of “direct communication”. You don’t need AMQP or JGroups for this.

Cheers,
Benoît

Hi Benoit,