I am just starting to take advantage of Sagas and have a few problems at the moment. One of which is an exception the other is the appearance of Domain Event Entry table.
What the code is attempting to do:
A @component handles an event titled SyncPointCreatedEvent this in turn schedules a Runnable which fires a command at some point in the future titled TriggerSyncPointCommand. This command’s @TargetAggregateIdentifier is transactionId.
A separate command handler titled SyncPointCommandHandler is picking up the command (TriggerSyncPointCommand) and converting it to a SyncPointTriggeredEvent and pushing it to the event bus. Finally we move into Saga code called SyncSaga. @StartSaga decorates a function for the aforementioned event SyncPointTriggeredEvent with an associatedProperty set to “transactionId”. This code appears to execute correctly and proceeds to perform the expected HTTP request (domain logic) and parse the response…
I believe this set up is correct but likely has a fundamental flaw given what happens next …
Exceptions:
The first thing that can be observed is an exception in the log (ConversionException). See log full log at the bottom of this post. One thing I don’t understand is why there is a reference to XStream when all events should be serialized with JSON as per configuration. For reference here is my AxonConfiguration:
@Configuration
@AutoConfigureAfter(AxonAutoConfiguration::class)
class AxonFrameworkConfiguration {
private val log: Logger = LoggerFactory.getLogger(this.javaClass)
@Autowired
fun registerInterceptors(commandBus: CommandBus) {
commandBus.registerDispatchInterceptor(BeanValidationInterceptor())
}
@Bean
fun snapshotterFactoryBean() = SpringAggregateSnapshotterFactoryBean()
@Bean
fun eventSchemaFactory(): EventTableFactory {
return PostgresEventTableFactory.INSTANCE
}
@Bean
fun eventSchema(): EventSchema {
return EventSchema()
}
@Bean
fun sagaSqlSchema(): SagaSqlSchema {
return PostgresSagaSqlSchema()
}
}
Domain Event Entry and Saga Entry Tables:
The other observation is that sagaentry table is entirely empty so it doesn’t appear that the associated id is working as expected.
The events are being recorded inside domainevententry table but the type is being written in as null .
Lastly, and this may not be related to Axon (at least it’s not clear to me at the moment) … I am seeing far more events for SyncPointTriggeredEvent than is actually being executed.
Fixed the serialization to be jackson by updating configuration in application.yml but I’m still observing a flood of events from a single triggered event with the type set to null as well so no activity inside the saga entry.
org.axonframework.serialization.SerializationException: Unable to serialize object
at org.axonframework.serialization.json.JacksonSerializer.serialize(JacksonSerializer.java:119) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.modelling.saga.repository.jpa.AbstractSagaEntry.(AbstractSagaEntry.java:58) ~[axon-modelling-4.0.jar:4.0]
at org.axonframework.modelling.saga.repository.jpa.SagaEntry.(SagaEntry.java:41) ~[axon-modelling-4.0.jar:4.0]
at org.axonframework.modelling.saga.repository.jpa.JpaSagaStore.createSagaEntry(JpaSagaStore.java:330) ~[axon-modelling-4.0.jar:4.0]
at org.axonframework.modelling.saga.repository.jpa.JpaSagaStore.insertSaga(JpaSagaStore.java:298) ~[axon-modelling-4.0.jar:4.0]
at org.axonframework.modelling.saga.repository.AnnotatedSagaRepository.storeSaga(AnnotatedSagaRepository.java:217) ~[axon-modelling-4.0.jar:4.0]
at org.axonframework.modelling.saga.repository.AnnotatedSagaRepository.lambda$doCreateInstance$3(AnnotatedSagaRepository.java:139) ~[axon-modelling-4.0.jar:4.0]
at org.axonframework.messaging.unitofwork.MessageProcessingContext.notifyHandlers(MessageProcessingContext.java:71) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.messaging.unitofwork.BatchingUnitOfWork.lambda$notifyHandlers$2(BatchingUnitOfWork.java:155) ~[axon-messaging-4.0.jar:4.0]
at java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:899) ~[na:1.8.0_162]
at org.axonframework.messaging.unitofwork.BatchingUnitOfWork.notifyHandlers(BatchingUnitOfWork.java:155) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.messaging.unitofwork.AbstractUnitOfWork.changePhase(AbstractUnitOfWork.java:222) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.messaging.unitofwork.AbstractUnitOfWork.commitAsRoot(AbstractUnitOfWork.java:83) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.messaging.unitofwork.AbstractUnitOfWork.commit(AbstractUnitOfWork.java:71) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.messaging.unitofwork.BatchingUnitOfWork.executeWithResult(BatchingUnitOfWork.java:111) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.eventhandling.AbstractEventProcessor.processInUnitOfWork(AbstractEventProcessor.java:136) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.eventhandling.TrackingEventProcessor.processBatch(TrackingEventProcessor.java:258) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.eventhandling.TrackingEventProcessor.processingLoop(TrackingEventProcessor.java:181) ~[axon-messaging-4.0.jar:4.0]
at org.axonframework.eventhandling.TrackingEventProcessor$TrackingSegmentWorker.run(TrackingEventProcessor.java:661) [axon-messaging-4.0.jar:4.0]
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.run(TrackingEventProcessor.java:771) [axon-messaging-4.0.jar:4.0]
at org.axonframework.eventhandling.TrackingEventProcessor$CountingRunnable.run(TrackingEventProcessor.java:588) [axon-messaging-4.0.jar:4.0]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.springframework.transaction.annotation.AnnotationTransactionAttributeSource and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: com.aramark.ess.command.domain.sagas.SyncSaga[“synchostService”]->com.aramark.ess.SyncHostServiceImpl$$EnhancerBySpringCGLIB$$7df569a8[“advisors”]->org.springframework.transaction.interceptor.BeanFactoryTransactionAttributeSourceAdvisor[0]->org.springframework.transaction.interceptor.BeanFactoryTransactionAttributeSourceAdvisor[“advice”]->org.springframework.transaction.interceptor.TransactionInterceptor[“transactionAttributeSource”])
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:312) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.ObjectArraySerializer.serializeContents(ObjectArraySerializer.java:249) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.ObjectArraySerializer.serialize(ObjectArraySerializer.java:210) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.ObjectArraySerializer.serialize(ObjectArraySerializer.java:22) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1017) ~[jackson-databind-2.9.6.jar:2.9.6]
at org.axonframework.serialization.json.JacksonSerializer.serialize(JacksonSerializer.java:114) ~[axon-messaging-4.0.jar:4.0]
… 21 common frames omitted
did you put the transient keyword in the fields that hold infrastructure components?
Axon serializes Saga state, and you wouldn’t want to serialize these components…
Also, consider using the XStream serializer for sagas. It doesn’t have any specific requirements on the structure of the class.
Regarding the tables not being filled, did you configure the Jpa/Jdbc SagaStore? If not, Sagas are stored in memory.
Yes that was the initial problem. Solved with JSONIgnore. I’ll attempt XStream serialization again.
I may have not configured the SagaStore only defined the schema.
Thank you for your input; I’ll attempt again tomorrow.
Can you tell me why I was seeing the event responsible for @StartSaga fired again and again? When I refactored the code to not utilize a saga the events did not behave this way.
My guess is that you have a tracking processor configured, but both the token and the saga are stored in memory. After a reboot, this will trigger redelivery of all events to the sagas.
Oh I see! Where can I read more about the delivery mechanism and does it work the same way with Aggregates? How much does the choice of message broker such as Kafka impact the delivery mechanics?
it’s not so much the delivery mechanism that causes this, but the fact that Axon doesn’t have a tracking token and want to start streaming from the beginning. Whether you’d use an Event Store, Kafka or anything else, will not change this behavior.
Still experiencing this behavior and now within an Aggregate and not a Saga. I’m having a bit of a hard time grasping the documentation due to gaps in understanding.
Tracking event processors, unlike subscribing ones, need a token store to store their progress in.
It’s not clear to me what the difference is between a subscribing one and an event processor. Are Aggregates event processors?
We’ve been able to demo Axon in the past without explicitly defining the token store, I believe this is due to the Spring Boot Autoconfiguration. As per the documentation:
Otherwise, if an EntityManager is available, the JpaTokenStore is defined.
I can also see two rows inside token_entry table (fyi we’re on a postgres database). These two entries are:
com.aramark.ess.query.domain (I don’t believe we explicitly defined this)
and sync-point which does come from a @ProcessingGroup(“sync-point”). The latter is what appears to be triggering the numerous number of events.
I did not see anything else in the documentation that suggested anything beyond having a token store. I’m not clear if the token has to be manually updated or notified for consumption in anyway.
As a final note, segment is 0, owner is null for both entries in the token store. The token type is org.axonframework.eventhandling.GapAwareTrackingToken
I’ve heard Gap mentioned in other questions but do not have a firm grasp on what it means.
Thank you for your continued help and support Allard. Axon is truly a wonderful framework but the learning curve for myself and our team is somewhat steeper than expected.
you should not have to worry about updating tokens. The fact that there are entries in the tokenStore, and that there are values inside the fields containing the token, is a good sign. The fact that it’s a GapAwareTrackingToken just means that you’re using the EmbeddedEventStore with a JPA or JDBC storage engine.
What underlying database are you using? When a tracking processor starts, it will check for the presence of tokens and continue from there. If there aren’t any tokens, it will initialize one (or more, depending on the number of initial threads the processor is configured for).
We see the replay behavior in different scenarios. For example, when Spring Boot/JPA is configured to spring.jpa.hibernate.ddl-auto, which defaults to create-drop if Spring Boot detects an embedded database.
Can you share a your configuration, so I can check if there’s anything unusual in there?
We’ve sold the problem and I apologize for leading you astray but it did not have anything to do with Axon. Unfortunately I don’t think sharing the solution will benefit others in the community,
I did get a better understanding of tokens through this conversation so thank you for responding.