Axon Java AxonServerCommandBus configuration as distributed command bus

Hi All,
I am trying to setup default distributed command bus to implement a orchestrated saga across 3 applications.Source code is available at

There are 3 independent services

  • OrderService port 9990
  • Payment Service port 9991
  • Shipping Service 9992

I have enabled default axon distributed command bus by setting below properties in application properties for all 3 apps.

  • axon.distributed.enabled=true
  • axon.distributed.load-factor=100

When I invoke http://localhost:9990/api/orders POST with request body { “itemType”:“LAPTOP”,
“price”:24,
“currency”:“USD”
}

It should call CreateOrderCommand and saga should call CreateInvoiceCommand running in payment service.

How ever delegation to remote service is not happening it throws exception
2021-04-15 20:03:01.588 WARN 27923 — [ault-executor-2] o.a.c.gateway.DefaultCommandGateway : Command ‘com.saga.orderservice.commands.CreateInvoiceCommand’ resulted in org.axonframework.commandhandling.NoHandlerForCommandException(No Handler for command: com.saga.orderservice.commands.CreateInvoiceCommand)

Why commands are not delegating to remote application?

Thanks,
Isuru

Hi Isuru,

where does the Saga live? In which of the applications? Did you configure the DistributedCommandBus in that application too?
If you were to send the CreateInvoiceCommand from the API directly, does it get executed normally? If not, it might be that the Invoice application doesn’t have the DistributedCommandBus configured correctly.

Hi Allard,
I have the Saga in OrderService app and it will orchestrate order service,payment service and shipment service.I have configured distributed command bus simply by putting below 2 properties in all 3 apps by referring to axon documentation.
axon.distributed.enabled=true
axon.distributed.load-factor=100
Is this sufficient to configure distributed command bus?

And to invoke a distributed command from saga running in order service i am using the below code piece.
commandGateway.send(new CreateInvoiceCommand(paymentId, orderCreatedEvent.orderId));

Is it the right way to invoke a distributed command?

Please clarify.There are no code examples in google for these.

Thanks,
Isuru

@Steven_van_Beelen
Do you have any input on how to enable distributed command bus?

I have configured distributed command bus simply by putting below 2 properties in all 3 apps by referring to axon documentation.
axon.distributed.enabled=true
axon.distributed.load-factor=100
Is this sufficient to configure distributed command bus?

Thanks,
Isuru

When using the DistributedCommandBus, you will also require a selection of one of Axon’s Extensions which provide the support to distributed those messages.
This means you will need to include either Axon’s JGroups Extension or the Spring Cloud Extension. The latter option will further require you to select a Spring Cloud Discovery implementation you prefer.

The Reference Guide page on the matter has actually quite recently been updated, so please take a look there too → https://docs.axoniq.io/reference-guide/axon-framework/axon-framework-commands/infrastructure#the-command-bus-distributed

Hope this clarifies your options @isuru!

Hi Stevan,
Thanks for the reply.I 'll check with Jgroup and do some r n d since it is axon implementation and discovery can be omitted.

Regards
Isuru

Hi @Steven_van_Beelen ,
I referred this axon jgroup documentation to enable distributed command bus.

As stated there I added below dependency to all projects .
implementation group: ‘org.axonframework’, name: ‘axon-spring-boot-starter-jgroups’, version: ‘3.4.3’

Also I added default springboot configurations for jgroup as follow and started all 3 apps successfully.

# enables Axon to construct the DistributedCommandBus
axon.distributed.enabled=true

# defines the load factor used for this segment. Defaults to 100
axon.distributed.load-factor=100

# the address to bind this instance to. By default, it attempts to find the Global IP address
axon.distributed.jgroups.bind-addr=GLOBAL

# the port to bind the local instance to
axon.distributed.jgroups.bind-port=7800

# the name of the JGroups Cluster to connect to
axon.distributed.jgroups.cluster-name=Axon

# the JGroups Configuration file to configure JGroups with
axon.distributed.jgroups.configuration-file=default_tcp_gossip.xml

# The IP and port of the Gossip Servers (comma separated) to connect to
axon.distributed.jgroups.gossip.hosts=localhost[12001]

# when true, will start an embedded Gossip Server on bound to the port of the first mentioned gossip host.
axon.distributed.jgroups.gossip.auto-start=false

How ever still I am getting “org.axonframework.commandhandling.NoHandlerForCommandException”
When sagamanager in orderservicer is trying to execute a remote jvm running command in payement service.

Source code is at GitHub - isuru-sam/axon-distributedcommandbus

Do you have a tip to fix this?Am I missing a configuration for jgroups?
I tried axon.distributed.jgroups.gossip.auto-start=true also.But same error.
Do I need to run TCP gossip servers manually?If so where to find a gossip server?
Please let me know your suggestion.
Thanks,
Isuru

I am guessing it might be a versioning issue. I base this on the fact you refer to 3.4.3 of the dependency, which is the latest Axon Framework 3 version.

Axon Framework has been on 4 for quite some time now though, with the most recent version being 4.5.
What would be beneficial to note next to this, is that the JGroups Extension is a separate repository from Axon Framework. For reference, here’s framework repository, and here the JGroups version. Note that both share the latest release as well, which we recommend you would use too.

So for now, would you please verify the dependency versions you are pulling in? Sticking to 4.x versions for both Framework and any Extensions is recommended, as those are compatible. Whenever you mix some 4.x and 3.x versions, I would expect start-up problems to occur.

@Steven_van_Beelen
https://mvnrepository.com/artifact/org.axonframework/axon-spring-boot-starter-jgroups/4.0-M2

This is the most recent jgroup starter dependency available.How to put the dependency for 4.5?
Anyways I put updated 4.5 dependancies.

implementation group: ‘org.axonframework’, name: ‘axon-spring-boot-starter’, version: ‘4.5’
implementation group: ‘org.axonframework.extensions.jgroups’, name: ‘axon-jgroups’, version: ‘4.5’

All 3 applications started succesufully But still remote commands cannot be executed?

It is ok to take some time if you are busy to check the source code at GitHub - isuru-sam/axon-distributedcommandbus

Please check when you are not not busy on the missing things in configuring distributed command bus with jgroups.This will help others to refer since this is a public repository and there are no good examples in the web as well.

Thanks
Isuru

It is under the org.axonframework group identifier, but it is not under the org.axonframework.extensions identifier. Here’s a link to the most recent version on Maven Central. For further clarity, here’s the pom.xml as you can spot it in the JGroups Extension repository I shared earlier.

Give that version a try and let us hear whether it solves the problem!

Adding this entry “implementation ‘org.axonframework.extensions.jgroups:axon-jgroups-spring-boot-starter:4.5’” started all 3 servers with detailed info like below

GMS: address=X556UQ-53040, cluster=Axon, physical address=172.17.0.1:7802
-------------------------------------------------------------------
2021-04-30 18:12:20.254  INFO 12390 --- [           main] org.jgroups.protocols.pbcast.GMS         : X556UQ-53040: no members discovered after 3001 ms: creating cluster as coordinator
2021-04-30 18:12:20.303  INFO 12390 --- [           main] o.a.e.j.c.JGroupsConnector               : Local segment (X556UQ-53040) joined the cluster. Broadcasting configuration.
2021-04-30 18:12:20.3

But still I cant execute remote commands I am gettign below error

NoHandlerForCommandException(No node known to accept [com.saga.orderservice.commands.CreateInvoiceCommand])

I have not implemented a separate @Configuration class inside spring app.Do I need to implement any configuration class to configure distributed command bus?

But as per documentation adding properties to the property file enough.

Please see whether you have a tip

Thanks
ISuru

Honestly, I’d have to debug your application at this stage to see what configuration is missing on your end. Maybe you can provide a sample project for us and others to look at to see what you’re missing?

All of this should work out of the box in a completely default Axon Config environment (so only with auto-configuration dependencies). Sorry for any inconvenience this might be causing.

As you are just looking for command distribution, there are two basic things you can try out:

  • Use Axon Server Standard Edition with the default configuration.
  • Check out Axon’s Spring Cloud Extension sample here, and expand it with your required pieces of code.

Hi Steven,
Here is the source code GitHub - isuru-sam/axon-distributedcommandbus.
OrderService owns the ordermanagement saga.When it tries to invoke CreateInvoiceCommand which is running on invoiceservice in different jvm command is not detected.Please try to debug it whenever you are free.

Thanks,
Isuru