Apologized ahead because I am a total noob in Axon framework. Here is what I need to achieve… I have an Axon framework application that I need to connect to (its event bus) and receive events and process them.
If you need to know the version of various pieces, just ask because it is too much to post here.
When I startup my spring boot app I did not get any errors, here is a small part of the startup logs:
restartedMain] com.feexx.reporting.config.AxonConfig : Setting up TrackingEventProcessors for threads, batch size and other configurations... annotated with @ProcessingGroup...
restartedMain] org.reflections.Reflections : Reflections took 75 ms to scan 1 urls, producing 3 keys and 3 values
restartedMain] com.feexx.reporting.config.AxonConfig : Configuring EventHandlerProjector Bean 'packageRecordProjector' with maxThreads: 20 and batchSize: 10.
restartedMain] com.feexx.reporting.config.AxonConfig : .. 'packageRecordProjector' successfully configured with processing group 'Package-Record'.
restartedMain] c.f.reporting.config.ApiSecurityConfig : Entering configure
restartedMain] o.s.s.web.DefaultSecurityFilterChain : Will secure Ant [pattern='/**'] with [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@75fa125e, org.springframework.security.web.context.SecurityContextPersistenceFilter@44a75ea3, org.springframework.security.web.header.HeaderWriterFilter@273df092, org.springframework.web.filter.CorsFilter@2c9dfe45, org.springframework.security.web.authentication.logout.LogoutFilter@52820678, com.feexx.reporting.config.ApiKeyAuthFilter@32299497, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@38bac51d, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@3df73d5e, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@25ea9ed1, org.springframework.security.web.session.SessionManagementFilter@22153964, org.springframework.security.web.access.ExceptionTranslationFilter@45b5769d, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@2d8cfe89]
restartedMain] o.s.b.d.a.OptionalLiveReloadServer : LiveReload server is running on port 35729
restartedMain] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
ckage-Record]-0] o.a.e.TrackingEventProcessor : Worker assigned to segment Segment[0/0] for processing
ckage-Record]-0] o.a.e.TrackingEventProcessor : Dispatching new tracking segment worker: TrackingSegmentWorker{processor=Package-Record, segment=Segment[0/0]}
ckage-Record]-1] o.a.e.TrackingEventProcessor : Fetched token: MergedTrackingToken{lowerSegmentToken=MergedTrackingToken{lowerSegmentToken=IndexTrackingToken{globalIndex=209300}, upperSegmentToken=IndexTrackingToken{globalIndex=209300}}, upperSegmentToken=IndexTrackingToken{globalIndex=209300}} for segment: Segment[0/0]
restartedMain] i.a.a.c.impl.AxonServerManagedChannel : Requesting connection details from axonserver:8124
restartedMain] i.a.a.c.impl.AxonServerManagedChannel : Successfully connected to axonserver:8124
restartedMain] i.a.a.connector.impl.ControlChannelImpl : Connected instruction stream for context 'default'. Sending client identification
restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8085 (http) with context path '/reporting-etl'
restartedMain] c.f.reporting.ReportingEtlApplication : Started ReportingEtlApplication in 11.918 seconds (JVM running for 12.809)
The spring boot app uses some custom code to hook into the Axon server. I will post it if anyone wants to see it.
So I went into the axon dashboard app and see our main axon app, the axon server, my spring boot app both connected. But when I generate an event from the axon app, I see the event dispatched into the handler methods of the axon app, but I did not see anything on the spring boot app as if no events was fired (in other words, the spring boot app did not see any events). I can also find the events in the axon dashboard (search) but I just don’t see any indications that it is dispatched and handled in the spring boot app.
Could you all give me some hints on how to perform diagnostics to check if the event bus and event handling methods in my spring boot app actually works? Any ideas is appreciated.
Here is the log, not super helpful. We are running the axon server within a container. If you know how to get more detailed logs, let me know and I can repost. If you look at the last 2 lines, it is saying recover-api-client (our main app) and reporting-etl-client (our springboot app) are both connected. I did initiate a begin and end event and nothing is showing in the log.
Regards,
Ed.
Listening for transport dt_socket at address: 8000
16:56:02.891 [main] INFO org.springframework.core.KotlinDetector - Kotlin reflection implementation not found at runtime, related features won't be available.
_ ____
/ \ __ _____ _ __ / ___| ___ _ ____ _____ _ __
/ _ \ \ \/ / _ \| '_ \\___ \ / _ \ '__\ \ / / _ \ '__|
/ ___ \ > < (_) | | | |___) | __/ | \ V / __/ |
/_/ \_\/_/\_\___/|_| |_|____/ \___|_| \_/ \___|_|
Standard Edition Powered by AxonIQ
version: 4.2.4
2023-01-23 16:56:03.939 INFO 9 --- [ main] io.axoniq.axonserver.AxonServer : Starting AxonServer on ee3234e1bb78 with PID 9 (/opt/axonserver/axonserver.jar started by root in /opt/axonserver)
2023-01-23 16:56:03.943 INFO 9 --- [ main] io.axoniq.axonserver.AxonServer : No active profile set, falling back to default profiles: default
2023-01-23 16:56:04.332 WARN 9 --- [kground-preinit] org.springframework.web.HttpLogging : For Jackson Kotlin classes support please add "com.fasterxml.jackson.module:jackson-module-kotlin" to the classpath
2023-01-23 16:56:10.396 INFO 9 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8024 (http)
2023-01-23 16:56:12.347 WARN 9 --- [ main] o.f.c.internal.database.base.Database : Flyway upgrade recommended: H2 1.4.199 is newer than this version of Flyway and support has not been tested.
2023-01-23 16:56:18.708 INFO 9 --- [ main] io.axoniq.axonserver.grpc.Gateway : Axon Server Gateway started on port: 8124 - no SSL
2023-01-23 16:56:19.205 INFO 9 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8024 (http) with context path ''
2023-01-23 16:56:19.210 INFO 9 --- [ main] io.axoniq.axonserver.AxonServer : Started AxonServer in 16.192 seconds (JVM running for 17.978)
2023-01-23 17:33:55.091 INFO 9 --- [pool-5-thread-5] i.a.a.logging.TopologyEventsLogger : Application connected: recover-api, clientId = recover-api-client, context = default
2023-01-23 17:36:33.825 INFO 9 --- [ool-5-thread-11] i.a.a.logging.TopologyEventsLogger : Application connected: reporting-etl, clientId = reporting-etl-client, context = default
Indeed, the log file does not contain any indication of the “disappearance” of events. The description of your issue has many similarities with my concern, with the only difference that in the log, I have this information which explains why my services do not receive events, but I cannot solve this problem
I have enabled logging in my axonserver docker container and I saw the exact error you are seeing! My events are blacklisted just like yours. Let me dig around a little and see if I can find out why my events are blacklisted.
That’s a kind of good news as it has been reproduce from another person’s project. I have tried disabling the default settings with this for my microservices consumers of the event blacklisted
Blacklisting events is just a result of no event handler was found to handle the event. It is just the result but the cause (of the problem) is still the fact that event handlers registered are not firing to handle the event.
I had a hard time believing that it is that buggy because the axon server is not new and few people ran into this issue.
So tomorrow I will concentrate on looking at how to properly configure the event handlers for spring boot apps.
I have no idea why your code works and stopped working all of a sudden. Like I said, according to the docs, blacklisting events means no handler was found to handle the events causing blacklisting. I also found out that sometimes, the events did not get blacklisted and yet no handlers was found handling them. As in your case what causes it to stop all of a sudden, I have no idea. We used the same term “blacklist” in another way. Is it possible some of your “TenantRegistration” went amock or had errors and fail somewhere causing blacklisting?
To close this topic, I was using a mixed case string for @ProcessingGroup annotation. After changing the value to an all lowercase string, events are now showing up in the handler methods.
