we decided to go for Axon for our project. The infrastructure uses Rancher for service discovery, so I am currently writing a DiscoveryClient that utilizes Rancher Metadata API to pull meta information about all service instances currently active in the rancher cluster.
However I have a problem to route commands to other instances in the cluster. I receive the exception that there couldnt be found any matching node for the command.
The problem is that the SpringCloudCommandRouter expects some metadata properties in the ServiceInstances returned from DiscoveryClient, that I cannot provide (because it is not configured/configurable in Rancher). The metadata properties are to find out what segments accept which kind of commands by specifying filters.
We want to configure this independent of what is configured on the ServiceInstance in the ServiceDiscovery provider, i. e. we want to set it once for the whole cluster to achieve something like this: MicroserviceA-Cluster accepts all commands from other MicroserviceA instances. We achieve this by letting the DiscoveryClient ask Rancher only for instances of the same type, i. e. the DiscoveryClient would only return ServiceInstances for MicroserviceA, if MicroserviceA is calling Rancher.
Maybe I also get the concept a bit wrong, but currently it looks like that I have to write some kind of hack, or to even write my own CommandRouter to get it working.
That sounds kind of problematic, because the SpringCloudCommandRouter is currently definitely utilizing the Metadata field of the ServiceInstance.
When the DistributedCommandBus receives updates of which commands it can handle, it propagates this knowledge by updating the membership of it’s CommandRouter.
In the case of the SpringCloudCommandRouter, that call will first update the ‘local service instance’ from the DiscoveryClient by pushing the load factor, the Command Filter and the class name of the Command Filter into the local instance it’s Metadata field.
This is thus also were other DiscoveryClients will retrieve knowledge from the other members in the system.
I’m not to familiar with Rancher to be fair, but might it be possible to propagate the Cluster-level metadata on to all it’s containing nodes?
That way you could still utilize the SpringCloudCommandRouter as is, since each instance would be aware of the cluster’s metadata.
in Rancher every container can have additional metadata yes. However we don’t have the necessity of commandFilter for investigating what services accept which commands, because Rancher organizes Microservices of the same type in so called “stacks”.
MicroserviceA-stack
MicroserviceA-Container1
MicroserviceA-Container2
MicroserviceA-Container3
MicroserviceB-stack
MicroserviceB-Container1
MicroserviceB-Container2
MicroserviceB-Container3
Now, when a Request from the frontend hits any instance of MicroserviceA, let’s say Container3, then the command router already knows all instances of MicroserviceA and only MicroserviceA. Every instance should be able to accept all commands for the same type.
Therefore I implemented a new CommandRouter. Basically it is almost similar to the SpringCloudCommandRouter, however it doesn’t work with Metadata and in the end just passes a AcceptAll commandFilter into the
public ConsistentHash with(Member member, int loadFactor, Predicate<? super CommandMessage<?>> commandFilter)
method. This was pretty easy and fast to achieve.
Works like a charm from my current POV and currently I am testing some use cases with the Command Bus.
Aah alright, that doesn’t sound to unreasonable in this scenario.
Great that a minor modification to the SpringCloudCommandRouter resulted in something which works for you, that’s definitely valuable to know.
Thanks for your solution-reply and good luck with the POV.
