We’re observing very strange behavior where an event processor with a given consumer group and multiple clients is not balancing events across all the nodes just the first (leader). I don’t believe this is normal Kafka consumer behavior, can someone please educate me on why this is happening.
To be honest, I am unsure whether you’re posing an Axon question or a Kafka question.
If it’s the latter, then I am afraid this is the wrong forum to post your request on; the Kafka/Confluent team is far better suited to explain balancing between nodes I think.
If this is an Axon question (which I am unsure off given the form of your request), then you’re obviously at the right place.
I’d like to point out that the Kafka-Extension is still in a release candidate state.
When it was still a part of Axon Framework 3.x we did not have a clear way of pointing this out as it was part of the same release cycle as the entire framework.
As of Axon Framework 4.x, all the extensions have been moved out of the core, thus giving them their own release cycle.
Any how, I am pointing this out because if you are expecting certain advanced behavior from the Kafka-Extension, that might be a thing which needs to be resolved still.
Lastly, when it comes to distributing any type of message, it’s far simpler to tie in an Axon Server instance, as Axon Server has the smarts to understand how different types of messages (like commands, events and queries) should be routed differently from one another. This will safe you lots of configuration time I’d think.