Axon Server message delivery system implementation

Hello,
I have a few questions about the message delivery system implementation in Axon Server.
I understand there are different routing mechanisms for each message type, i.e. different mechanisms for commands, queries and events.

Here follow some questions I have. Could you also mention whether the implementation differs for each message type, or whether the implementation is the same?:

  1. Is the communication pub/sub?

  2. Do the messages always arrive at the message handlers in the order they were published? Is there a queue of some sort?

  3. Does an event handler have to be registered with Axon Server before a message can be published? I.e. does the message go to some persistent storage (like the queue in question 2), or do messages get lost?

  4. Is there acknowledgement between Axon Server and message handlers? If so, is there also a retry mechanism and what strategy does this support: at least once delivery, or exactly once delivery?

  5. Is gRPC the default technology used for messaging between “message publishing applications”/“message receiving applications” and Axon Server?

First and foremost, welcome to the forum, @sandervanthul :wave:

Now, let me go through your question, one by one:

  1. For events, you could say it is, yes. For commands and queries, the answer is no. For events, an app would “tell” Axon Server it is interested in events, starting from a specific token (or, offset, if you use Kafka-terms). This opens a gRPC stream between Axon Server and the client. Whenever Axon Server receives new events, it pushes them to all the subscribed clients.
  2. By default, the answer to this is yes. However, for commands and queries, you can define a priority value. Assuming you are using Axon Framework, that is! For queries for example, you can define a QueryPriorityCalculator, which reacts on a QueryMessage, expecting the implementer to return a value. Axon Framework then contains a queue, taking insert order and this priority into account.
  3. Axon Server will always store your events. By default, indefinitely even. It does so, as it is not only a Message Router, but also an Event Store. From a theoretically stance, you would never throw away any events from an Event Store. However, if there’s some level of redundancy on the events (lets say, after 5 years or so), you can define a so-called context to be ephemeral.
  4. Yes, there is. And yes, you can define a retry behavior, although this is client specific again. And for commands only at this point in time. AF5 will have retry behavior for queries as well, by the way. This is not in place for events, as Axon Server is an Event Store. If the client is unable to publish and thus store an event, the fact simply didn’t happen. Hence, your state didn’t change, so subsequent commands and queries would never be able to act on a none-existent state change. The “strategy” as you call it is at least once delivery. Hence, we recommend you make your message handler idempotent.
  5. Yes, gRPC is the default communication technology in Axon Server.

If you want to get some additional input on Axon Server messaging approach, you can also check the follow links:

  1. Podcast with Milan and Sara on Message Routing Protocols in Axon Server
  2. Webinar by Allard on Message Routing in Axon Server

Let me know if this suffices, @sandervanthul!

Thank you very much Steven for this comprehensive answer, this has been very helpful!

One quick follow up on question 3. Namely, what is the behavior when command and queries are send but the handlers for them are not available? Are they persisted until the handler becomes available (perhaps in a queue), or might they be lost?

Sure thing @sandervanthul, glad to help.

Axon Server will notice no command- or query-handler present for the dispatched command/query. As a result, a NoHandlerForCommandException or NoHandlerForQueryException is thrown by the Axon Framework application.

If you want to be resilient to this, with a level (I assume) of retry behavior until the (micro)service servicing the command/query is back, that would be custom-built for Axon Framework or Axon Server.

In the former, I’d wager a CommandBus or QueryBus decorator pattern may suffice. For the latter, you’d have to build an Axon Sever plugin. I would personally go for the former, but that’s mostly based on where my focus lies with AxonIQ (which is Axon Framework development).