I’m running an app with Axon 3.4.3, spring-boot 2.3.2, Java 11, and postgresql. Occasionally I am seeing a saga loaded after it has completed and been deleted from the DB. This only occurs when two events are processed at nearly the same time. The behavior is as follows:
Expected
- thread1: puts event on event bus
- thread2: puts event on the event bus
- thread1: Saga is loaded and calls SagaLifecycle.end()
- thread2 drops event because the saga has ended and was deleted from DB
Actual
- thread1: puts event on event bus
- thread2: puts event on the event bus
- thread1: Saga is loaded and calls SagaLifecycle.end()
- thread2: Saga is loaded and calls SagaLifecycle.end()
This has been problematic as the two events triggering the end of the saga issue different commands and thus result in different business logic.
Debugging the AnnotatedSagaRepository I can see the deleteSaga method called and then a subsequent doLoadSaga resulting in a response from the DB. If I query the DB directly the saga isn’t there at this point.
A couple other things worth noting
- this app runs on a single node (for now)
- the properties contain “spring.jpa.open-in-view=false”
- handling of the events is wrapped using Springs @Transactional annotation
- changing the @Transactional annotation isolation level to serializable results in “PSQLException: ERROR: could not serialize access due to concurrent update” on the second attempt to delete the saga entry
I would expect the transaction of the first load to commit before the second load begins but it appears there is indeed concurrent access to the same saga. Is this expected behavior? Perhaps I have something configured incorrectly? Any help or guidance would be appreciated.