Constant reconnect on QueryChannelImpl

Axon Server
#1

I connected a service with the axon server. It just uses the query gateway to request data from QueryHandlers also connected to the axon server … this works as expected.

But the log is polluted with (logs multiple times per second):

2020-10-29 12:04:25.222 INFO 72812 --- [@ets-20181509-0] i.a.a.c.query.impl.QueryChannelImpl : QueryChannel connected, 0 registrations resubscribed

Seems I am missing some configuration? I am not even using subscription queries yet …

Thanks
Jan

#2

Hi Jan,
the message about “registrations subscribed” is actually about the handler methods that are (re)subscribed to AxonServer. It has nothing to do with Subscription Queries.

If you have a component that only sends Queries, but doesn’t handle them, the value will always be 0.
It will also be 0 at first startup when the connection is established, but the handlers still need to be registered. In that case, you will see logging statements that mention the handlers being subscribed.

I am wondering what causes the reconnections all the time, though. How do you configure the connection to AxonServer?

#3

Hi Allard,

we are using the axon-spring-boot-starter with the following (unsuspicious) configuration:

axon:
  axonserver:
    token: "..."
    context: "..."
    servers: localhost
    ssl-enabled: false

What we did, though, and this could be where we must adopt st: We came up with the concept of “operation modes” (see discussion in https://github.com/AxonFramework/AxonFramework/issues/1256).

So for a system, that only should use the queryGateway, we disabled the JPAAutoConfiguration, Event- and CommundBus, like this:

@EnableAutoConfiguration(exclude = JpaAutoConfiguration.class)
public class QueryOnlyModeConfiguration implements QueryOnlyMode {

    @Bean
    @Primary
    public SagaStore noopSagaStore() {
        return NoOperation.SAGA_STORE;
    }

    @Bean
    @Primary
    public CommandBus noopCommandBus() {
        return NoOperation.COMMAND_BUS;
    }

    @Bean
    @Primary
    public EventStore noopEventBusAndStore() {
        return NoOperation.EVENT_STORE;
    }
}

(where NoOperation… throws UnsupportedOperationExceptions on all functions).

This works, in the sense that we a) can get the query result via queryGateway (and see the service in the axon server overview) and b) do not get tables (eventStore, sagaStore, tokenStore, …) created in our DB. I guess, we missed some internal requirements and should at least provide an inMemory implementation of … what? Any idea? Where does the QueryChannelImpl check if I am registered/subscribed?

Thanks
Jan

#4

The configuration looks all right, actually. The problem doesn’t seem to be there.

I am wondering where all these log items would come from. Could you perhaps do a stack dump each time the log entry is called (you can make the IDE do that instead of stopping at a breakpoint). I wonder where the requests to connect come from. I would normally just expect one at startup, and no more.

#5

Trying to … but don’t see how this can be done … got a hint for intellij idea for me?

#6

Hi Jan. More than happy to teach you the trick that one of my colleagues taught me :slight_smile:

  • When you set a breakpoint in IntelliJ, right click it.
  • In the pop-up you get, you can click a link “More”. That opens the breakpoint window, with the newly created breakpoint selected.
  • In the right panel, at the top, disable the option “suspend”
  • In that same panel, a little further down, enable the options “Log: ‘breakpoint hit’ message” and “stacktrace”.

Run and have fun :slight_smile:

#7

Hi @allardbz . I did not use the IDEA trick yet … but still got an update: I suppressed the log-info, until we got a call because we had 100ks erroneous requests in instana (around 1 every 80 ms) … In the instana report, you can see, that it is a permission problem:

Type RPC Client
Category rpc
Flavor grpc
Host xxx.xy
Remote Port 8124
Procedure/Method io.axoniq.axonserver.grpc.query.QueryService/OpenStream

Error
PERMISSION_DENIED: Invalid token for io.axoniq.axonserver.grpc.query.QueryService/OpenStream

We then did a try and error, the problem stopped, after giving the “I am only interested in using the query gateway” system the access-right: “READ”. Funny thing: this is marked as deprecated in the docs …

for sake of completeness these are the access rights in total now (panning to drop commands/events soon):

ourContext:DISPATCH_COMMANDS
ourContext:DISPATCH_QUERY
ourContext:READ_EVENTS
ourContext:READ

I must admit, I am a bit confused … if the READ right seems to be that important, why is it not active by default … and why is it deprecated? and what should we use instead?

I will hand this over to one of my colleagues that can file support requests, just copying this here to round up the discussion an in case it is of value for other users as well …

yours
Jan

#8

Hi Jan,

that last message had the info I needed to find out what was happening. The new connector is quite “pro-active” in setting up a query stream, even if that stream isn’t strictly needed. Unfortunately, it now hits the authorization limitation.

Adding the authorization to HANDLE_QUERIES (and potentially also HANDLE_COMMANDS, for the same reason) should resolve it.

I’ll register a fix for this in the connector’s Github Repository

1 Like
#9

Hm … the “HANDLE_XY” authorizations you mentioned are neither documented nor selectable in the axon-server UI …
How should I proceed?

#10

My bad, they’re called SUBSCRIBE_QUERY_HANDLER and SUBSCRIBE_COMMAND_HANDLER.
Not sure where I got the other names from :frowning:

#11

Ah, ok, thanks … will try.

Naming is a bit confusing … SUBSCRIBE_QUERY_HANDLER seems to mean “I can subscribe (=consume) to a query handler” … but reads like “i have a query handler and am allowed to provide (=register this at the server)” … so the active/passiv role seems to be inverted.

#12

Hi Jan,

actually, it exactly means what you read from it: ‘you can subscribe a query handler’. The reason that you need to add this role is a limitation of the connector, which actively connects the query stream, even if there is no handlers actually registered.
A fix for this is on its way.

1 Like