[Kafka Extension] Kafka Publisher does not support Axon Framework 4.5.5

Hi there,

I have a question regarding the usage of the Kafka extension (4.5) together with the most recent Axon framework version (4.5.5).

As far as I can see there was recently a change in the Axon framework (commit 6f8f593) (4.5.4+) which broke the KafkaPublisher.java builder of the mentioned Kafka extension.
Parts of the problem are already addressed it in this fix / commit ca833db but the publisher seems to be still broken.

So I would be interested if you already have plans to release a fix or if it will take some time. I really don’t want to push you with this post - it’s all about helping us to decide if we can continue on integrating the extension or if we have to stay with our custom event handler sending messages to Kafka for now.

Thanks you for your great work at the Axon framework. We really appreciate it!

Hi @martin, good morning.

You are right and I’ve fixed the link to the correct commit which fixed this issue.
The problem, as I can see, is that the commit was made towards master branch and not 4.5.x so we would have to merge that into 4.5.x and make a follow up release of the extensions-kafka.

But prior to do that, can you verify using the 4.6.0-SNAPSHOT version if the problem is gone?
That would make sure everything will work out without any other problem.

KR,

Thanks @lfgcampos
I’ll try to test it later today. Is there a public available 4.6.0-SNAPSHOT of the Kafka extension or do I need to build it by myself?
Just in addition to the fix ca833db… I might be wrong but even it is only on master I think the KafkaPublisher is not fixed yet. I’ll double check later.

Hi @martin,

All of our SNAPSHOTs build are published to Sonatype so you can get it from there.
It should be easy to add it as as server to your settings.xml.

Double checking the KafkaPublisher, you could be right as I can still see the ‘old’ XStream piece of code here.
But the Builder also give us the option to override that here. Nevertheless, a fix should be made!

Are you up for a PR? No pressure of course and we will work on that for sure, probably together with a new release as soon we get your input about the tests, to make sure everything is included!

KR,

Hi @lfgcampos,

I’ve used the Sonatype repository as you suggested and pulled the latest available build of version 4.6.0-SNAPSHOT (20211124).

Unfortunately I can confirm that the KafkaPublisher is still throwing an exception due to the mentioned XStream problem (The XStream instance is a hard requirement and should be provided).

Additionally using the builder method to override the message converter does not help. The error will be thrown as soon as you initialize the builder because the root cause XStreamSerializer.builder().build() is part of an initial class variable assignment.

I could give it a try to create a pull request by myself and use the same fix used in ca833db.

Thanks for your help!

2 Likes

Hey @martin, thank you for the update!

Correct, this need to be fixed indeed.
If you are up for a PR, that would be great - feel free to also create an issue for that on the repo or I can do it for you if needed.

If you need any other help, feel free to ping me.

KR,

Hi @martin, just coming here to thank you for the PR!

Also, we made a new release for this extension this morning… it should pop up soon on maven central!

Kr,

Hi All,

When I follow the PR, it seemed like this made it to the 4.5.1 however I still get the same error. Am I missing something?

org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘streamableKafkaMessageSource’ defined in class path resource [org/axonframework/extensions/kafka/autoconfig/KafkaAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.axonframework.extensions.kafka.eventhandling.consumer.streamable.StreamableKafkaMessageSource]: Factory method ‘streamableKafkaMessageSource’ threw exception; nested exception is org.axonframework.common.AxonConfigurationException: The XStream instance is a hard requirement and should be provided

@lfgcampos Sorry I’ve missed your message here. Thanks a lot for the quick release! We are already successfully using it.

@Sanjay_Deshpande That might be based on a little misunderstanding from my side. My commit was fixing the remaining parts for the “event publishing” side of the extension. An older commit was already covering the others - also from the “event consuming” side (ca833db). I’ve only rebased my commit to the release branch but it did not include ca833db. Our use case is only producing Kafka events with the extension, so it works for us.

@lfgcampos If you like I could also backport ca833db on the weekend.

Aah no worries there, @martin, we’ve got it covered.
Lucas’ on vacation but nudged me about this.

As the commit you referenced is mine, I thought it fair to cherry-pick that to the patch branch.
Furthermore, I’ve taken the liberty to immediately do another release of the Kafka Extension (which you can find here.

Thus, no effort is needed anymore from your end.
Nonetheless, the gesture is much appreciated, of course. :slight_smile:

FYI, @Sanjay_Deshpande, look out for using the 4.5.2 release of the Kafka Extension!
Do note that even though Axon’s logic will pick up the security issues pestering XStream, it definitely does so in a fashion that isn’t inherently recommended for end users.
We introduced the fix simply to not impose a breaking change on our users.

As such, you should see warning signs when starting up your application to immediately set your own XStream instance for the XStreamSerializer.
Thus, I highly recommend configuring that within your application to stay compliant.

1 Like

@martin , @Steven_van_Beelen, You guys are awesome. Thank you for the quick turnaround. Greatly appreciated.

@Steven_van_Beelen your advice duly noted.

Have a Happy New year.

1 Like