No reconnect of AxonQueryBus after the AxonServer restarts

Hello!

Need help to understand is it a bug or normal behavior on AxonServer restart action.
We have the following env:

  • AxonServer 4.3.1 (running in k8s stateful set. manifest was used from the official doc)
  • Spring Boot app with axon-spring-boot-starter 4.3.1
    The situation by steps(for running instance of AxonServer):
  1. Spring Boot application starts with the following logs:

12:44:24 [main] INFO o.a.a.c.AxonServerConnectionManager - Requesting connection details from localhost:8124

12:44:25 [main] INFO o.a.a.c.AxonServerConnectionManager - Reusing existing channel

12:44:25 [main] INFO o.a.a.c.AxonServerConnectionManager - Re-subscribing commands and queries

12:44:25 [main] INFO o.a.a.c.query.AxonServerQueryBus - Creating new query stream subscriber

12:44:25 [main] INFO o.a.a.c.command.AxonServerCommandBus - Creating new command stream subscriber

  1. In Axon dashboard I’m able to see available list of commands and queries. Then AxonServer restarts
  1. Spring Boot app becomes reconnect to AxonServer, and when connection is established prints the following logs:

12:53:01 [AxonServerConnectionManager-0] INFO o.a.a.c.AxonServerConnectionManager - Re-subscribing commands and queries

12:53:01 [AxonServerConnectionManager-0] INFO o.a.a.c.command.AxonServerCommandBus - Resubscribing Command handlers with AxonServer

12:53:01 [AxonServerConnectionManager-0] INFO o.a.a.c.command.AxonServerCommandBus - Creating new command stream subscriber

Here we didn’t see a message like “Creating new query stream subscriber”, so after successful reconnection, there are no queries in the Axon dashboard. Since I’m no able to push queries via queryGateway, because no @QueryHandler is registered after restart.

P.S with command subscribers, everything is good. It shows commands in the dashboard and handlers still working.

Also, I’ve looked through the sources of AxonServerCommandBus and AxonServerQueryBus and found some differences in “SubscriberObserver”

  1. AxonServerCommandBus.CommandProcessor.getSubscriberObserver#onError(line 792) - has line “subscriberStreamObserver = null;”, which means, that new observer will be created on next connection.
  2. AxonServerQueryBus.QueryProcessor.getSubscriberObserver#onError(line 721) - doesn’t have “setting to null” the observer. So, it never re-creates the observer.

Is this a normal behavior of error handler mechanisms? :slight_smile:

Hi Artsiom,

I have the same issue without k8s. My applications periodically gets disconnected from Axonserver SE for unknown reason. When connected again there are no query handlers anymore. I think you can create an issue for that.

Kind regards,
Aleksey.

I’ve created an issue

Hi, we have the same issue.

And this is a real problem. Our axon server got restarted by a server patch and everything failed after that!

We will try 4.3.2

Hi all,

The issue referenced and created by Artisiom has been resolved and added to Axon Framework 4.3.2.
If you are thus noticing a similar problem in your application, it’s recommended to first update to 4.3.2 or higher to see if it mitigates the problem.

Cheers,
Steven