Route Axon events to different Event Hubs (Kafka Topics)

Axon Framework
, ,
1

Hello,

In my app, I use Event Hub with the Kafka API as an event bus. I have several questions that I’m looking for answers to currently:

  1. How can I route Axon events to different Kafka topics?
  2. What number of Event Hub partitions should I consider for Axon?

I’m using
implementation ‘org.axonframework.extensions.kafka:axon-kafka-spring-boot-starter’

and have following configuration

axon:
  kafka:
    bootstrap-servers: ...
    default-topic: axon_events
    message-converter-mode: cloud_event
    properties:
      security.protocol: SASL_SSL
      sasl.mechanism: OAUTHBEARER
      sasl.jaas.config: org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;
      sasl.login.callback.handler.class: com.azure.spring.cloud.service.implementation.kafka.KafkaOAuth2AuthenticateCallbackHandler
    publisher:
      processing-group: ...
    producer:
      event-processor-mode: pooled_streaming
2

You can configure a topic resolver to determine the topic based o the event message. When you return an empty optional, it’s not sent at all.

I’m not sure about the partitions. One partition will most likely be enough, but if you want to parallelize, you need more. I like 24 since it’s not insanely high yet allows equal distribution of partitions for many instance counts.

2 Likes
3

Thanks a lot, it’s what I was looking for.

One more question though, regarding consumer groups.

Assuming I want to have a few consumer groups, I need to create SubscribableMessageSource with different groupId, right? Are there some alternatives?

4

Yes, each source can only have one consume group. However, I wonder why you need this?