I recently enabled OpenTelemetry for an application. Something I noticed was that lots of errors started popping up related to CachingEventSourcingRepository.load:
The aggregate was not found in the event store
However, when looking at the regular error logging (Prometheus) these errors don’t seem to appear. I’m not sure whether this is an issue or not as the application seems to behave just fine. I was wondering how I could prevent these errors from popping up? Any thoughts?
Here’s the full stack trace as logged by OpenTelemetry:
at org.axonframework.eventsourcing.EventSourcingRepository.doLoadWithLock(EventSourcingRepository.java:131)
at org.axonframework.eventsourcing.CachingEventSourcingRepository.doLoadWithLock(CachingEventSourcingRepository.java:120)
at org.axonframework.eventsourcing.CachingEventSourcingRepository.doLoadWithLock(CachingEventSourcingRepository.java:40)
at org.axonframework.modelling.command.LockingRepository.doLoad(LockingRepository.java:137)
at org.axonframework.modelling.command.LockingRepository.doLoad(LockingRepository.java:60)
at org.axonframework.modelling.command.AbstractRepository.lambda$null$6(AbstractRepository.java:141)
at java.util.HashMap.computeIfAbsent(Unknown Source)
at org.axonframework.modelling.command.AbstractRepository.lambda$load$8(AbstractRepository.java:140)
at org.axonframework.tracing.Span.runSupplier(Span.java:134)
at org.axonframework.modelling.command.AbstractRepository.load(AbstractRepository.java:137)
at org.axonframework.modelling.command.AggregateAnnotationCommandHandler$AggregateCommandHandler.handle(AggregateAnnotationCommandHandler.java:568)
at org.axonframework.modelling.command.AggregateAnnotationCommandHandler$AggregateCommandHandler.handle(AggregateAnnotationCommandHandler.java:557)
at org.axonframework.messaging.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:57)
at be.cegeka.vconsult.notification.infrastructure.axon.ExceptionWrappingHandlerInterceptor.handle(ExceptionWrappingHandlerInterceptor.java:17)
at org.axonframework.messaging.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:55)
at org.axonframework.messaging.unitofwork.DefaultUnitOfWork.executeWithResult(DefaultUnitOfWork.java:77)
at org.axonframework.commandhandling.SimpleCommandBus.lambda$handle$3(SimpleCommandBus.java:198)
at org.axonframework.tracing.Span.run(Span.java:72)
at org.axonframework.commandhandling.SimpleCommandBus.handle(SimpleCommandBus.java:189)
at org.axonframework.commandhandling.SimpleCommandBus.doDispatch(SimpleCommandBus.java:163)
at org.axonframework.commandhandling.SimpleCommandBus.dispatch(SimpleCommandBus.java:130)
at org.axonframework.extensions.jgroups.commandhandling.JGroupsConnector.processDispatchMessage(JGroupsConnector.java:339)
at org.axonframework.extensions.jgroups.commandhandling.JGroupsConnector.lambda$receive$7(JGroupsConnector.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Seeing the stack trace, the exception is triggered by a command coming into your JGroupsConnector.
Whenever this catches an exception, it should wrap that as a reply to the node that dispatched the command.
So, perhaps straightforward, but have you checked the logs of your other instances?
Thanks for your response.
Meanwhile I found out why this error message appears in the traces and not in logs. It’s due to a bad legacy implementation that exists in the code base, which is not related to Axon. I have some refactoring ahead of me
Maybe another question: is there a way to reliably know whether a particular aggregate exists before sending a command to it, given a JGroups distributed command bus scenario? Or whenever it doesn’t exist, somehow a create command can be sent before the actual command itself?
Perhaps we need to rethink how the aggregate needs to be created related to the current commands.
I think the @CreationPolicy annotation is your best bet here, @JanVanRyswyck.
You can pair that annotation together with the @CommandHandler annotation, providing either of the following policies:
ALWAYS
CREATE_IF_MISSING
NEVER
So, by annotating a command handler with @CreationPolicy(AggregateCreationPolicy.CREATE_IF_MISSING), the framework will construct the aggregate if an aggregate for the provided aggregate identifier does not exist.
Although this is a very powerful tool, it does come with a performance penalty.
Your Repository will need to check all the aggregate identifiers in the Event Store before it can be certain the provided identifier does not exist.
Hence, depending on the size of the store, a create-if-missing can be quick or slow.
Regardless of speed, the performance penalty is higher than knowing before hand that the aggregate does (not) exist.
Concluding, I think this will solve your predicament, @JanVanRyswyck. But, use this option with caution would be my recommendation.
@CreationPolicy(AggregateCreationPolicy.CREATE_IF_MISSING) ,
This one actually worked, but why shouldn’t it be an aggregate created automatically when service is started or may be whenever its first command gets executed? Why do we need to mention it explicitly?
Firstly, my apologies for the late reply here, @Manoj.
I’ve been on an extended vacation, but am happily back to make Axon Framework awesome
Joking aside, let’s go to your question.
But, to be frank, I am not certain from what angle you’re coming with this.
If an Aggregate doesn’t exist, Axon Framework will create a new aggregate for it. In essence, this behavior reflects the creation policy AggregateCreationPolicy#ALWAYS.
However, ALWAYS is different from CREATE_IF_MISSING. The latter will construct an aggregate only if there is no instance yet with the given aggregate identifier.
So, to circle back to your question:
… but why shouldn’t it be an aggregate created automatically when service is started or may be whenever its first command gets executed?
I would wager the question you’re posing directs towards uniquely identified aggregates. However, it is that which isn’t clear from your question.
Hence why I am asking what “angle” you are coming from.
So, feel free to elaborate to clarify your question! With that, I should be able to give better guidance.