Detecting whether command was posted to command bus (AxonServer)

Axon Framework
1

Hello,

I’m designing an API that handles commands in an asynchronous way. But I’d like to respond to clients with an error if the command wasn’t ACKed by bus.

Is there any way to distinguish command dispatching failures from command handling failures? My initial understanding was that when CommandGateway.send(Object command) returns (even without waiting for CompletableFuture to complete), it means that message is already posted to command bus.

I got this assumption based on CommandBus javadoc (that is used by CommandGateway.send(Object command) behind the scenes)

/**
...
   When the method returns, the only guarantee provided by the CommandBus implementation is that the command has been successfully received
...
*/
<C, R> void dispatch(CommandMessage<C> command, CommandCallback<? super C, ? super R> callback)

But looks like this is not the case? Method returns without complaints when AxonServer is not running. What am I missing? Or this is by design and we shouldn’t really care about the stage at which command processing failed (dispatching vs handling)?

Kind regards,
Beka

2

In case you really want to react based on in which stage command has failed, the easiest way to achieve this is to use Reactor Extension.

Only this extension offers dispatch and result interceptors.

You could define error behavior if dispatch fails and if handler fails (result interceptor)

reactiveGateway.registerDispatchInterceptor(
    command -> command.onError...(...)
);

reactiveGateway.registerResultHandlerInterceptor(
    (command,result) -> result.onError...()...
);

API is very flexible, should fit all your needs but requires the usage and knowledge of the Reactor framework with Spring Webflux.

3

Thanks Stefan for the suggestion. Currently, I don’t see the need for the service I’m working on to be Reactive. So I wouldn’t want to switch to a different paradigm just for the sake of meeting the initial requirement in my previous post. I’d like to understand though why such an option isn’t available in non-reactive AxonFramework, is this by design?

Let’s say, we design the client in a way that it sends pre-validated commands and assumes that they will succeed, then I can understand that most likely error handling logic will be the same, regardless of the stage the error occurred at - notify the client/user. Maybe this was the intention behind such a design? I still feel a little uncomfortable though to ACK the command receipt and notify the failure later for the trivial case when the command bus is not reachable or is simply shut down.

Also could you please clarify the javadoc, do I interpret it correctly? Does it need to be changed?

Beka

4

If the command handler fails, an error will be wrapped inside CommandExecutionException, and exceptionally block on completable future will be called.

If command dispatching fails you should get an exception of type CommandDispatchException.

When using send(…) The way to catch these exceptions to use exceptionally block on CompletableFuture
commandGateway.send(...).exceptionally(t -> ...) , you should get you exception here.
If you want to catch an exception without this block you should use sendAndWait(…).

5

Maybe my question was formulated incorrectly. I don’t want to just differentiate command dispatching failures from handling failures. I want to differentiate them while keeping command handling asynchronous but dispatching synchronous. Your last suggestion I believe means that either command will be dispatched asynchronously (CompletableFuture.exceptionally ) or it will be handled synchronously (sendAndWait).

Anyway, I’m not trying to get a solution here, I wish to understand the background and design decisions made in AxonFramework with regards to this topic.

Thanks,
Beka

6

Is there any way to distinguish command dispatching failures from command handling failures?

Yes there is, as both scenarios are signaled as different types of exceptions.
If command dispatching fails, you should see a CommandDispatchException.
This scenario would occur if your CommandBus implementation is incapable of sending anything at all.

When command handling fails, thus your command already went through the bus, that’s when you should receive a CommandExecutionException instead.

Method returns without complaints when AxonServer is not running. What am I missing?

The fact you see everything working in absence of Axon Server, is likely because all your commands can flow through the so-called local segment. The AxonServerCommandBus is a distributed implementation of the CommandBus, which in turn uses this local segment for local command sending and handling.

Hope this clarifies things for you @Beka_Tsotsoria!

7

Hi Steven,

thank you for getting back to this topic! I still feel that I wasn’t clear enough, let me give another try.

I understand that we’ll get CommandDispatchException if we use

commandGateway.sendAndWait<Void>(command);

my question is if we get the same exception if we use this

commandGateway.send<Void>(command);

but without waiting on the returned CompletableFuture.

from the tests I’ve done, it looks like the answer is no.


The fact you see everything working in absence of Axon Server, is likely because all your commands can flow through the so-called local segment

My understanding is that AxonServer must be contacted first in any case, even if the command routing algorithm selects the JVM, which dispatched the command originally. I believe this is the source code responsible for that part.

If that’s true, I expect that dispatching should fail, and I believe it does, but as you can see on line 177, exceptions that may happen during the dispatching are reported in the same way as the command handling results i.e. via commandCallback. In that way, it is impossible to dispatch the command synchronously but handle it asynchronously, while detecting dispatching exceptions early i.e. without waiting for command handling result.

I hope this makes my point clear.

Kind regards,
Beka

8

My understanding is that AxonServer must be contacted first in any case, even if the command routing algorithm selects the JVM, which dispatched the command originally. I believe this is the source code responsible for that part.

I indeed agree with you but was making a guesstimate based on what I pulled from your description. I now notice that was a wrong assumption from my part, based on the following line in your original message:

Method returns without complaints when AxonServer is not running.

This to me sounds like everything worked fine, but I believe what you intended to share is that the CompletableFuture returned, albeit with an exceptional result.

Back to the actual thing you are investigating, sendAndWait() will bubble up the exception, as you have likely noticed. Internally sendAndWait() is nothing more than the send() operation, where the framework waits for the outcome of the CompletableFuture for you.
This indeed makes your assumption correct. Not checking the outcome of the CompletableFuture will simply not land you the CommandDispatchException directly in your face. You will instead have to check when the future is complete yourself if this is required. Or let another component, like Spring’s Web UI, resolve it for you.

I’d wager the definition of “received” in the documentation is what put you off in this case. I would say you did provide the command to the CommandBus inside your JVM by any means. Whether it went from one end to another, so if dispatching succeeded, is still up for debate.

Maybe we should turn this situation around, by going back to your original statement:

But I’d like to respond to clients with an error if the command wasn’t ACKed by bus.

Followed up by your first question:

Is there any way to distinguish command dispatching failures from command handling failures?

I would assume that either, regardless of whether you use send or sendAndWait on the CommandGateway, or dispatch(C) / dispatch(C, CommandCallback<C, R>) on the CommandBus, can be achieved by checking for the aforementioned exceptions in my previous response. The checking approach is just different:

  • CommandGateway#send(Object) - The exception from the CompletableFuture will be rethrown from, so you can add a try-catch if needed.
  • CommandGateway#sendAndWait(Object) - The outcome is present in the CompletableFuture, which should be validated in your set up
  • CommandBus#dispatch(C) - This would net you, sadly, nothing, as the CommandCallback used in this case is a no-op
  • CommandBus#dispatch(C, CommandCallback<C, R>) - The result is returned to the CommandCallback#onResult, where the CommandResultMessage should be validated.

By the way, if you feel the description is too ambiguous right now, any improvements are always fair game as pull requests of course. Furthermore, I hope this clarifies your option further @Beka_Tsotsoria. If anything else is still open, definitely reach out.

9

This to me sounds like everything worked fine, but I believe what you intended to share is that the CompletableFuture returned, albeit with an exceptional result.

No, I meant what literally CommandBus#dispatch javadoc says, that is when the method returns.

I’d wager the definition of “received” in the documentation is what put you off in this case. I would say you did provide the command to the CommandBus inside your JVM by any means. Whether it went from one end to another, so if dispatching succeeded, is still up for debate.

Exactly. I understood “received” as the receipt by the actual CommandBus (i.e. successful dispatch). Otherwise if method returned without exceptions, it obviously means that the command (argument) was “received” by CommandBus object.

In any case, I think the javadoc can be improved. I’ll try to submit the PR later today.

Now, getting back to the synchronous dispatch and asynchronous handling topic… I still don’t think I’d get what I want with the suggestions above. As per the contract of API I’m designing, successful response means that command was received, but not necessarily handled yet.

  • If the API layer responds with success only after calling CommandGateway#sendAndWait, then it means that the command will be handled synchronously in a successful scenario (which is not what I want).
  • If the API layer responds with success after calling CommandGateway#send, then there’s no guarantee that the command was actually dispatched to the bus.

That’s why I started to ask myself, do I really want to differentiate such dispatching and handling failures? Most likely they will be reported to the API consumer in the same, asynchronous, communication channel. But maybe there are other use cases too.

So in short, this topic boils down to the simple questions:

  • is it possible to dispatch commands synchronously, while handling them still asynchronously.
  • if not, what can be the practical benefits of having an additional option for dispatching commands synchronously
10

Awesome, thanks for coming back here on such short notice again @Beka_Tsotsoria. The confusion on the CommandBus is understandable. If I would have to give a reason why it is stated as such…pretty sure it is legacy. The CommandBus used to only reside within the JVM. As such, dispatching over some piece of infrastructure (like Axon Server) was not taken into account when that was drafted up.

Thus, thanks for spotting this ambiguity. And for providing the pull request; I’ll have a look at that as soon as my time allows it.

Indeed, let’s get back to your original question. Or simpler, the shortened versions you’ve shared at the end of your previous post:

Is it possible to dispatch commands synchronously while handling them still asynchronously?

The short answer to this is no. All the bus implementations mostly assume the entirety of sending (and for commands and queries, handling too) to be asynchronous. The outlier here is the old fashioned “inside the JVM” SimpleCommandBus, dating back from Axon 0.5. Theoretically, it could be done, if the communication layer would return an acknowledgement of receiving. Furthermore, the Command Response would have to be different to accompany different options. Whether there is a real benefit for this is up for debate.

If not, what can be the practical benefits of having an additional option for dispatching commands synchronously?

To be honest with you, my first input would tell me no. I tend to state that you should “embrace the asynchronous nature of a messaging system”, as it allows for complete decoupling (e.g. into distinct microservices) wherever you use them. Noted, this does not state that dispatching should be asynchronous too, as essentially you haven’t gone over the wire. But I find it hard to discover benefit in having the distinction between synchronous dispatching and asynchronous handling. Using the CompletableFuture as the single result containing both the process of dispatching and handling seems straightforward enough to me.

As stated earlier, anything is up for debate. Any input is always very much valued and might jus steer the process of any of Axon’s components. So, if you discover any benefits of making dispatching synchronous over using the current CompletableFuture’s API, please do share.

11

Thanks Steven for the clarifications. I agree. I began to change my view. The example that I mentioned earlier

I still feel a little uncomfortable though to ACK the command receipt and notify the failure later for the trivial case when the command bus is not reachable or is simply shut down.

represents quite an extreme case. And what’s more, CommandBus being down or having other persistent connectivity issue is much of a bigger problem (which should be addressed in the first place!) than the fact that API “lied” about the successful ACK of command.

1 Like
12

Happy to help out @Beka_Tsotsoria.
Think this is a valuable discussion which will benefit others who read this in the future as well.