Error when migrate to spring boot 3.2.0

Axon Framework
, ,
1

I used spring boot 3.1.5 with axon 4.9.0, when i migrate to 3.2.0 i get this error:


2023-11-23 18:39:31.754 | [main] | WARN |  |  | i.a.a.c.i.AxonServerManagedChannel |  Connecting to AxonServer node [localhost:8124] failed.
io.grpc.StatusRuntimeException: UNKNOWN
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
	at io.axoniq.axonserver.grpc.control.PlatformServiceGrpc$PlatformServiceBlockingStub.getPlatformServer(PlatformServiceGrpc.java:250)
	at io.axoniq.axonserver.connector.impl.AxonServerManagedChannel.connectChannel(AxonServerManagedChannel.java:115)
	at io.axoniq.axonserver.connector.impl.AxonServerManagedChannel.createConnection(AxonServerManagedChannel.java:335)
	at io.axoniq.axonserver.connector.impl.AxonServerManagedChannel.ensureConnected(AxonServerManagedChannel.java:300)
	at io.axoniq.axonserver.connector.impl.AxonServerManagedChannel.getState(AxonServerManagedChannel.java:227)
	at io.axoniq.axonserver.connector.impl.ContextConnection.ensureConnected(ContextConnection.java:237)
	at io.axoniq.axonserver.connector.impl.ContextConnection.connect(ContextConnection.java:165)
	at io.axoniq.axonserver.connector.AxonServerConnectionFactory.connect(AxonServerConnectionFactory.java:166)
	at org.axonframework.axonserver.connector.AxonServerConnectionManager.createConnection(AxonServerConnectionManager.java:130)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at org.axonframework.axonserver.connector.AxonServerConnectionManager.getConnection(AxonServerConnectionManager.java:126)
	at org.axonframework.axonserver.connector.command.AxonServerCommandBus.subscribe(AxonServerCommandBus.java:216)
	at org.axonframework.modelling.command.AggregateAnnotationCommandHandler.lambda$null$0(AggregateAnnotationCommandHandler.java:138)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1850)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.axonframework.modelling.command.AggregateAnnotationCommandHandler.subscribe(AggregateAnnotationCommandHandler.java:141)
	at org.axonframework.config.AggregateConfigurer.lambda$initialize$34(AggregateConfigurer.java:454)
	at org.axonframework.config.LifecycleOperations.lambda$onStart$0(LifecycleOperations.java:62)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:662)
	at org.axonframework.config.DefaultConfigurer.invokeLifecycleHandlers(DefaultConfigurer.java:1058)
	at org.axonframework.config.DefaultConfigurer.invokeStartHandlers(DefaultConfigurer.java:1004)
	at org.axonframework.config.DefaultConfigurer$ConfigurationImpl.start(DefaultConfigurer.java:1156)
	at org.axonframework.spring.config.SpringAxonConfiguration.start(SpringAxonConfiguration.java:76)
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:284)
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:467)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:256)
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:201)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:965)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:619)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:753)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:455)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:323)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1342)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1331)
Caused by: java.lang.UnsupportedOperationException: null
	at io.grpc.netty.AbstractHttp2Headers.isEmpty(AbstractHttp2Headers.java:40)
	at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:419)
	at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:352)
	at io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onHeadersRead(Http2InboundFrameLogger.java:56)
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader$2.processFragment(DefaultHttp2FrameReader.java:476)
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readHeadersFrame(DefaultHttp2FrameReader.java:484)
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:253)
	at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:159)
	at io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41)
	at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:188)
	at io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:393)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:453)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
2023-11-23 18:39:31.756 | [main] | INFO |  |  | i.a.a.c.i.AxonServerManagedChannel |  Failed to get connection to AxonServer. Scheduling a reconnect in 2000ms
1 Like
2

Hi Aymen,

Thanks for reporting this. I get the same error when upgrading to Spring Boot 3.2.0. We should have a solution soon, but for now, please move back to 3.1.x.

2 Likes
3

The problem is an incompatibility with Netty.
The only way around when using Spring Boot as the parent pom I found working was to set all the netty depencies like:

<dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-codec</artifactId>
            <version>4.1.79.Final</version>
</dependency>

explicitly. If you get the spring boot version via a bom, you should be able to just also add the netty bom:

<dependency>
              <groupId>io.netty</groupId>
              <artifactId>netty-bom</artifactId>
              <version>4.1.79.Final</version>
              <type>pom</type>
</dependency>

Should do the trick. A more neat/easy solution would be to include shaded netty dependencies to the connector, so this would not be a problem anymore.

4

The problem is introduced in Upgrade to Netty 4.1.101.Final · Issue #38447 · spring-projects/spring-boot · GitHub so everything 3.0.13+ is affected, 3.1.x has the same problem.

5

True, we already released a new connector, which solves the problem. Release Axon Server Connector for Java version 2023.2.0 · AxonIQ/axonserver-connector-java · GitHub we should have a new release of Axon Framework using this, as well as a new bom soon.

6

Hm, https://mvnrepository.com/artifact/org.axonframework/axon-server-connector lists no new versions since the new versions numbers were introduced?

7

It can be confusing, I agree. The one you linked has a dependency on the connector I mentioned. It’s a module in Axon Framework and should have a 4.9.1 release soon using the new version. The maven coordinates for the connector I mentioned can be found here: https://mvnrepository.com/artifact/io.axoniq/axonserver-connector-java. Since they are compatible you can already use this version. Or you can wait a bit for the new release of Axon Framework or the release of the axon bom.

8

I replaced org.axonserver/axon-server-connector with io.axoniq/axonserver-connector-java, but now things like

import org.axonframework.axonserver.connector.AxonServerConnectionManager
import org.axonframework.axonserver.connector.event.axon.AxonServerEventScheduler

are missing

Do I need some more jars, or have things changed and I need to refactor?

9

You don’t need to replace, the one from framework already depends on the other. So you need to update the dependency. That alone should be enough.

10

Ah. Success :slight_smile:

I’ll wait for the new bom until I move into production, but this is fine for now. Doing the monthly “upgrade all deps in the pom” thing…

:star:

1 Like
11

Hi @Gerard,
just to let you know:

I tried the workaround by adding

      <dependency>
        <groupId>io.axoniq</groupId>
        <artifactId>axonserver-connector-java</artifactId>
        <version>2023.2.0</version>
      </dependency>

to <dependencyManagement>, but application tests will fail with tons of Spring errors. (Details would not help here I guess).

It seems to work in a very simple test, but I get errors in many of my modules that for example deal with Axon projections and JPA.
I get a number of netty dependencies in my classpath from other libraries I am using (like io.projectreactor.netty:reactor-netty-http or AWS SDKs like software.amazon.awssdk:cognitoidentityprovider).

I have to stay with Spring Boot 3.1 and hope that you guys get a stable fix for without tweaking dependency versions.

Klaus

12

Unfortunately, the latest 3.1 (3.1.6) also has the problem. Something you could still try, is adding these dependencies:

        <dependency>
            <groupId>org.axonframework</groupId>
            <artifactId>axon-server-connector</artifactId>
            <version>4.9.1-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.axonframework</groupId>
            <artifactId>axon-messaging</artifactId>
            <version>4.9.1-SNAPSHOT</version>
        </dependency>

Or wait for the 4.9.1 AF release and 4.9.2 axon bom release, which should be there soon.

13

New version has been released, Axon Framework - Release 4.9.1.