Need help with understanding how snapshots are working in Axon and why it's brakes my aggregate

Hello folks!

I’m learning Axon and really need to understand how implement snapshotting in right way…

I’ve implemented a little chat API using axon + spring-boot. code is pretty simple and located here: https://github.com/daggerok/axon-snapshots

basically if Im using config like this:

@ComponentScan(basePackageClasses = [AxonApp::class])
class AxonConfig {

companion object {
const val amount = 4
}

// @Bean
// fun snapshotterFactoryBean() = SpringAggregateSnapshotterFactoryBean()
//
// @Bean(“axonChatRepository”)
// fun axonChatSnapshotterRepository(eventStore: EventStore, snapshotter: Snapshotter): Repository =
// EventSourcingRepository(ChatAggregator::class.java, eventStore, EventCountSnapshotTriggerDefinition(snapshotter, amount))

@Bean(“axonChatRepository”)
fun axonChatRepository(eventStore: EventStore): Repository =
EventSourcingRepository(ChatAggregator::class.java, eventStore)

@Bean
fun eventStorageEngine(serializer: Serializer, axonMongoTemplate: MongoTemplate): EventStorageEngine = MongoEventStorageEngine(
serializer, null, axonMongoTemplate, DocumentPerEventStorageStrategy())
// serializer, null, amount, axonMongoTemplate, DocumentPerCommitStorageStrategy())

@Bean
fun serializer(): Serializer = JacksonSerializer()

@Bean(“axonMongoTemplate”)
fun axonMongoTemplate(mongoClient: MongoClient, @Value("${spring.datasource.name}") name: String): MongoTemplate =
DefaultMongoTemplate(mongoClient, name)
.withDomainEventsCollection(“events”)
.withSnapshotCollection(“snapshots”)
}

everything is working fine, but if I will remove axonChatRepository and uncomment all commented lines, snapshot will be triggered right after 4th command and on 5th command my aggregator will fail with error

2018-02-10 04:37:24.470  WARN 76342 --- [nio-8080-exec-5] o.a.c.callbacks.LoggingCallback          : Command resulted in exception: daggerok.chat.LeaveRoomCommand

org.axonframework.eventsourcing.IncompatibleAggregateException: Aggregate identifier must be non-null after applying an event. Make sure the aggregate identifier is initialized at the latest when handling the creation event.
        at org.axonframework.eventsourcing.EventSourcedAggregate.publish(EventSourcedAggregate.java:160) ~[axon-core-3.1.2.jar!/:3.1.2]
        at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_152]
        at org.axonframework.eventsourcing.EventSourcedAggregate.lambda$initializeState$1(EventSourcedAggregate.java:212) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.inspection.AnnotatedAggregate.lambda$execute$2(AnnotatedAggregate.java:174) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.AggregateLifecycle.lambda$execute$3(AggregateLifecycle.java:199) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.AggregateLifecycle.executeWithResult(AggregateLifecycle.java:166) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.AggregateLifecycle.execute(AggregateLifecycle.java:198) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.inspection.AnnotatedAggregate.execute(AnnotatedAggregate.java:174) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.eventsourcing.EventSourcedAggregate.initializeState(EventSourcedAggregate.java:209) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.eventsourcing.EventSourcingRepository.doLoadWithLock(EventSourcingRepository.java:215) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.eventsourcing.EventSourcingRepository.doLoadWithLock(EventSourcingRepository.java:45) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.LockingRepository.doLoad(LockingRepository.java:162) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.LockingRepository.doLoad(LockingRepository.java:49) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.model.AbstractRepository.lambda$load$7(AbstractRepository.java:116) ~[axon-core-3.1.2.jar!/:3.1.2]
        at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[na:1.8.0_152]
        at org.axonframework.commandhandling.model.AbstractRepository.load(AbstractRepository.java:115) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.AggregateAnnotationCommandHandler$AggregateCommandHandler.handle(AggregateAnnotationCommandHandler.java:195) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.AggregateAnnotationCommandHandler$AggregateCommandHandler.handle(AggregateAnnotationCommandHandler.java:189) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.AggregateAnnotationCommandHandler.handle(AggregateAnnotationCommandHandler.java:151) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.AggregateAnnotationCommandHandler.handle(AggregateAnnotationCommandHandler.java:43) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.messaging.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:57) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.messaging.interceptors.CorrelationDataInterceptor.handle(CorrelationDataInterceptor.java:55) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.messaging.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:55) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.messaging.unitofwork.DefaultUnitOfWork.executeWithResult(DefaultUnitOfWork.java:69) ~[axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.SimpleCommandBus.handle(SimpleCommandBus.java:156) [axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.SimpleCommandBus.doDispatch(SimpleCommandBus.java:127) [axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.SimpleCommandBus.dispatch(SimpleCommandBus.java:91) [axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.gateway.AbstractCommandGateway.send(AbstractCommandGateway.java:79) [axon-core-3.1.2.jar!/:3.1.2]
        at org.axonframework.commandhandling.gateway.DefaultCommandGateway.send(DefaultCommandGateway.java:95) [axon-core-3.1.2.jar!/:3.1.2]
        at daggerok.chat.ChatCommandsResource.leaveRoom(ChatCommandsResource.java:40) [classes!/:0.0.1]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_152]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_152]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_152]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_152]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:667) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110) [spring-boot-actuator-1.5.10.RELEASE.jar!/:1.5.10.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106) [spring-boot-actuator-1.5.10.RELEASE.jar!/:1.5.10.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.14.RELEASE.jar!/:4.3.14.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_152]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_152]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.27.jar!/:8.5.27]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_152]

It doesn’t matter which command It’s currently handling, basically if I will use threshold 3, then commands previous command will fail.
As far as I understand, snapshotting require some additional configuration / implementation…

Can anyone of you explain me what Im doing wrong and how it should be designed?

If you have a chance to look on it, I’ve described in details in README how it can be repreduced.
Please, let me know if I have to put here more information

Thanks in any advice!

Hello again!

While no one answering I found main problem:

fun serializer(): Serializer = JacksonSerializer()

Do you know if it’s possible to replace XStreamSerializer with JacksonSerializer? If so, that I have to do to make it working properly?

OK, I was able to manage it by my own.

Just put it here if someone like me will looking for this important note

Even if XStreamSerializer worked fine, but it was store my serializable payload as XML, which is obviously useless with mongodb as well as more verbose… So I found that default ObjectMapper needed for JacksonSerializer has to be tuned for field visibility.

Anyway, here is worked Jackson configuration which is fixed my issue (maybe someone will add more concrete note regarding that into the documentation):


@Bean
fun objectMapper(): ObjectMapper {
  val objectMapper = ObjectMapper()
  objectMapper.setVisibility(FIELD, ANY)
  return objectMapper
}

@Beanfun serializer(): Serializer = JacksonSerializer(objectMapper()) //XStreamSerializer()

Hi Maksim,

Good to hear you were capable to find a solution on your own.

I do have some info to add about the Serializers in Axon though.

You’ve got to option to set three different Serializers in your application, namely:

  • the default Serializer

  • the messageSerializer

  • the eventSerializer

The default serializer is (of course) the default serializer within an Axon application, which will be used to serialize Events, Command/Query messages (in a Distributed set up), Snapshots, Tokens and Sagas.

For Snapshots, Tokens and Sagas we typically recommend to leave the default, the XStreamSerializer as is.

Yes it is more verbose, but it will allow you to serialize them regardless.

For (event) messages you’d typically want to use a different serializer, as you’ve already found out.

So in your example, that would mean only adjusting the event serializer, not the default and message one.

To do that, you can just add a Serializer Bean with the qualifier eventSerializer tied to it.

That way, only your events will be serialized as JSON, instead of everything.

If you’d however want your default Serializer to be the JacksonSerializer, then you’re obviously done already.

Hope this gives you some insights!

Cheers,

Steven

Hi Steven,

Thanks for response.

regarding different serialization options, did you mean these setting?

+axon.serializer.general=jackson
+axon.serializer.messages=java
+axon.serializer.events=jackson

from https://github.com/AxonFramework/AxonFramework/commit/59758590f7e8e502b4cd6d7864676905a74a1baa#diff-72be6eb09074e4dc850ed90c056cb9c1R17

and

axon.serializer.format=xml
axon.serializer.events.format=json

from here: https://github.com/AxonFramework/AxonFramework/issues/446

?

Previously I’ve seen these configurations in Axon github commits, but wasn’t able fully understand how to use them correctly. If i just updating these settings nothing is changing (or at lease thinks I need), unless I update java / spring-boot beans configuration

Hi Maksim,

Ah, so yes, that would be a nice thing to use for you, but that’s a feature for 3.2.

And, 3.2 isn’t released yet, so it makes sense it’s not working for you at the moment.

We’re planning to release 3.2 at the end of this month, so you should be able to use it around then.

Otherwise, doing this:

@Bean

@Qualifier(“eventSerializer”)
public Serializer eventSerializer() {

return new JacksonSerializer();

}

Should be enough to set the event serializer to use Jackson.

Hope this helps!

Cheers,

Steven