Replaying events with Kafka

Imported from Google Groups
1

Hi,

I’m using Axon 3.3.5 with spring-boot and I’m trying to replay events in order to get databases sync. What I did was deleting my query db to get all the events from command db again.

My query side is configured with:

axon:
serializer:
general: XSTREAM
eventhandling:
processors:
“[bankevents]”:
source: kafkaMessageSource
mode: TRACKING
kafka:
default-topic: “events”
consumer:
group-id: “bank-group”
bootstrap-servers:

  • localhost:9092

I tried to reset token in order to get it working, but nothing happens:

config.eventProcessorByProcessingGroup(“bankevents”, TrackingEventProcessor.class)
.ifPresent(trackingEventProcessor -> {
trackingEventProcessor.shutDown();
trackingEventProcessor.resetTokens();
trackingEventProcessor.start();
});

Any advice?

2

Same issue for me, looking for suggestions

3

I tried to delete token manually and didn’t work too.

4

Hi,

there seems to be an issue with the KafkaTrackingToken in combination with how Kafka wants to coordinate consumers. This is being addressed at the moment.

Kind regards,

Allard

5

Hi Allard,

Thanks for your replay.

Can you send me the git issue? Maybe I can contribute with my logs and testing it.

6

I tried the 3.4-SNAPSHOT today and the problem still persist there.

Can you send me the git issue? Maybe I can contribute with my logs and testing it.

7

Any help here?

8

Hi Gustavo,

the issue isn’t fixed yet in the master branch. It has been contributed by an external developer, so we’re also dependent on his availability. I’ll ping him for the progress.

Cheers,

Allard

9

Hello Allard,

It will be fixed with 4.0 version? Do you have the git issue, so I can track and contribute there.

10

Hi Gustavo,

in the Axon 4.0 release, we’ve moved the Kafka connector (as well as some other connectors) to different repositories. For Kafka, this is: https://github.com/AxonFramework/extension-kafka. There is currently one open issue, which covers the replay issue as well.

The KafkaConnector has not yet been released as 4.0, as we want this issue to be resolved first.

Cheers,

Allard

11

So I am assuming this issue is fixed in version 4 as I am not seeing it in the issue list anymore. I do have a question about this KafkaTrackingToken, is it just any offset of Kafka partition , or in case of multiple partitions on the topic, how is it achieving the tracking since Kafka does not guarantee ordering among different partitions.

12

The Kafka related issues have moved to the Kafka extention’s repository:
https://github.com/AxonFramework/extension-kafka/issues

Ordering across partitions will not be guaranteed. Axon will do a best-effort ordering based on timestamp.

Hope this helps.
Cheers,

13

Thanks for confirming this. I totally understand the difficulties across partitions as this is Kafka’s limitation.