TrackingEventProcessor - Could not initialize segments. Some segments were already present

Hi,

I’m using Axon 4.5 and AxonServer.
The database is MS SQL Server. I initialized the DB with Flyway.
My whole setup is in my github repo.

The AxonConfiguration is here, but I also included it below.

When I start up the application, this error (see below) keeps on popping up. I already deleted content of all tables etc.

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.5)

2021-05-25 20:42:45.015  INFO 1849 --- [  restartedMain] b.a.ce.calendar.CalendarMgmtApplication  : Starting CalendarMgmtApplication using Java 14.0.2 on ce-calendar-5b47886d68-mc54d with PID 1849 (/usr/src/app/calendar-mgmt/target/classes started by root in /usr/src/app/calendar-mgmt)
2021-05-25 20:42:45.018  INFO 1849 --- [  restartedMain] b.a.ce.calendar.CalendarMgmtApplication  : No active profile set, falling back to default profiles: default
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/root/.m2/repository/org/codehaus/groovy/groovy/2.5.14/groovy-2.5.14.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2021-05-25 20:42:45.377  INFO 1849 --- [  restartedMain] o.s.b.devtools.restart.ChangeableUrls    : The Class-Path manifest attribute in /root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-osgi-2.2.10.jar referenced one or more files that do not exist: file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-core-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-xjc-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-impl-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-jxc-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb1-impl-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-runtime-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-core-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/jaxb-api-2.2.12-b140109.1041.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/txw2-2.2.10.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/istack-commons-runtime-2.19.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/stax-ex-1.7.7.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/FastInfoset-1.2.13.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/msv-core-2013.6.1.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/xsdlib-2013.6.1.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/relaxngDatatype-20020414.jar,file:/root/.m2/repository/com/sun/xml/bind/jaxb-osgi/2.2.10/isorelax-20090621.jar
2021-05-25 20:42:45.377  INFO 1849 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2021-05-25 20:42:45.377  INFO 1849 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : For additional web related logging consider setting the 'logging.level.web' property to 'DEBUG'
2021-05-25 20:42:47.814  INFO 1849 --- [  restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-05-25 20:42:47.862  INFO 1849 --- [  restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 41 ms. Found 0 JPA repository interfaces.
2021-05-25 20:42:48.108  INFO 1849 --- [  restartedMain] o.s.c.a.ConfigurationClassPostProcessor  : Cannot enhance @Configuration bean definition 'org.axonframework.springboot.autoconfig.EventProcessingAutoConfiguration' since its singleton instance has been created too early. The typical cause is a non-static @Bean method with a BeanDefinitionRegistryPostProcessor return type: Consider declaring such methods as 'static'.
2021-05-25 20:42:48.370  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#66962b94#34' of type [org.axonframework.spring.config.annotation.SpringContextParameterResolverFactoryBuilder$ClasspathParameterResolverFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.370  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#66962b94#34' of type [org.axonframework.messaging.annotation.MultiParameterResolverFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.373  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#4bfaae50#34' of type [org.axonframework.spring.config.annotation.SpringBeanDependencyResolverFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.374  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#61bc05aa' of type [org.axonframework.spring.config.annotation.SpringBeanParameterResolverFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.630  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '__axon-parameter-resolver-factory' of type [org.axonframework.spring.config.ApplicationContextLookupParameterResolverFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.630  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '__axon-parameter-resolver-factory' of type [org.axonframework.messaging.annotation.MultiParameterResolverFactory] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.646  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#69a39961' of type [org.axonframework.spring.config.annotation.SpringHandlerDefinitionBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.647  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#69a39961' of type [org.axonframework.messaging.annotation.MultiHandlerDefinition] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.649  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#975488f' of type [org.axonframework.spring.config.annotation.SpringHandlerEnhancerDefinitionBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.650  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '(inner bean)#975488f' of type [org.axonframework.messaging.annotation.MultiHandlerEnhancerDefinition] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:48.655  INFO 1849 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean '__axon-handler-definition' of type [org.axonframework.messaging.annotation.MultiHandlerDefinition] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-05-25 20:42:49.386  INFO 1849 --- [  restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2021-05-25 20:42:49.401  INFO 1849 --- [  restartedMain] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-05-25 20:42:49.401  INFO 1849 --- [  restartedMain] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.45]
2021-05-25 20:42:49.591  INFO 1849 --- [  restartedMain] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-05-25 20:42:49.591  INFO 1849 --- [  restartedMain] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 4213 ms
2021-05-25 20:42:51.048  INFO 1849 --- [  restartedMain] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-05-25 20:42:51.485  INFO 1849 --- [  restartedMain] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-05-25 20:42:51.499  INFO 1849 --- [  restartedMain] b.a.c.c.CalendarMgmtApplication$1        : Database startup detected after 0.451 seconds
2021-05-25 20:42:51.551  INFO 1849 --- [  restartedMain] b.a.ce.calendar.CalendarMgmtApplication  : ######## Target database calendar already exists; skipping creation
2021-05-25 20:42:56.632  INFO 1849 --- [  restartedMain] com.zaxxer.hikari.HikariDataSource       : HikariPool-2 - Starting...
2021-05-25 20:42:56.733  INFO 1849 --- [  restartedMain] com.zaxxer.hikari.HikariDataSource       : HikariPool-2 - Start completed.
2021-05-25 20:42:56.799  INFO 1849 --- [  restartedMain] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-05-25 20:42:56.868  INFO 1849 --- [  restartedMain] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.4.30.Final
2021-05-25 20:42:57.070  INFO 1849 --- [  restartedMain] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-05-25 20:42:57.208  INFO 1849 --- [  restartedMain] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.SQLServerDialect
2021-05-25 20:42:58.339  INFO 1849 --- [  restartedMain] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-05-25 20:42:58.348  INFO 1849 --- [  restartedMain] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-05-25 20:42:58.947  INFO 1849 --- [  restartedMain] o.f.c.internal.license.VersionPrinter    : Flyway Community Edition 7.7.0 by Redgate
2021-05-25 20:42:59.006  INFO 1849 --- [  restartedMain] o.f.c.i.database.base.DatabaseType       : Database: jdbc:sqlserver://my-mssql-mssql-linux:1433;maxResultBuffer=-1;sendTemporalDataTypesAsStringForBulkCopy=true;delayLoadingLobs=true;useFmtOnly=false;useBulkCopyForBatchInsert=false;cancelQueryTimeout=-1;sslProtocol=TLS;jaasConfigurationName=SQLJDBCDriver;statementPoolingCacheSize=0;serverPreparedStatementDiscardThreshold=10;enablePrepareOnFirstPreparedStatementCall=false;fips=false;socketTimeout=0;authentication=NotSpecified;authenticationScheme=nativeAuthentication;xopenStates=false;sendTimeAsDatetime=true;trustStoreType=JKS;trustServerCertificate=false;TransparentNetworkIPResolution=true;serverNameAsACE=false;sendStringParametersAsUnicode=true;selectMethod=direct;responseBuffering=adaptive;queryTimeout=-1;packetSize=8000;multiSubnetFailover=false;loginTimeout=30;lockTimeout=-1;lastUpdateCount=true;encrypt=false;disableStatementPooling=true;databaseName=calendar;columnEncryptionSetting=Disabled;applicationName=Microsoft JDBC Driver for SQL Server;applicationIntent=readwrite; (Microsoft SQL Server 14.0)
2021-05-25 20:42:59.009  WARN 1849 --- [  restartedMain] o.f.c.i.database.base.DatabaseType       : Microsoft SQL Server 14.0 does not support setting the schema for the current session. Default schema will NOT be changed to ACERTA !
2021-05-25 20:42:59.092  INFO 1849 --- [  restartedMain] o.f.core.internal.command.DbValidate     : Successfully validated 3 migrations (execution time 00:00.031s)
2021-05-25 20:42:59.111  INFO 1849 --- [  restartedMain] o.f.core.internal.command.DbMigrate      : Current version of schema [ACERTA]: 100.2
2021-05-25 20:42:59.113  INFO 1849 --- [  restartedMain] o.f.core.internal.command.DbMigrate      : Schema [ACERTA] is up to date. No migration necessary.
2021-05-25 20:42:59.244  INFO 1849 --- [  restartedMain] org.quartz.impl.StdSchedulerFactory      : Using default implementation for ThreadExecutor
2021-05-25 20:42:59.246  INFO 1849 --- [  restartedMain] org.quartz.simpl.SimpleThreadPool        : Job execution threads will use class loader of thread: restartedMain
2021-05-25 20:42:59.259  INFO 1849 --- [  restartedMain] org.quartz.core.SchedulerSignalerImpl    : Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
2021-05-25 20:42:59.260  INFO 1849 --- [  restartedMain] org.quartz.core.QuartzScheduler          : Quartz Scheduler v.2.3.2 created.
2021-05-25 20:42:59.262  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Detected usage of MSSQLDelegate class - defaulting 'selectWithLockSQL' to 'SELECT * FROM {0}LOCKS WITH (UPDLOCK,ROWLOCK) WHERE SCHED_NAME = {1} AND LOCK_NAME = ?'.
2021-05-25 20:42:59.262  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Using db table-based data access locking (synchronization).
2021-05-25 20:42:59.264  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : JobStoreCMT initialized.
2021-05-25 20:42:59.265  INFO 1849 --- [  restartedMain] org.quartz.core.QuartzScheduler          : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'MY_JOB_SCHEDULER' with instanceId 'NON_CLUSTERED'
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
  Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore' - which supports persistence. and is not clustered.

2021-05-25 20:42:59.266  INFO 1849 --- [  restartedMain] org.quartz.impl.StdSchedulerFactory      : Quartz scheduler 'MY_JOB_SCHEDULER' initialized from an externally provided properties instance.
2021-05-25 20:42:59.266  INFO 1849 --- [  restartedMain] org.quartz.impl.StdSchedulerFactory      : Quartz scheduler version: 2.3.2
2021-05-25 20:42:59.266  INFO 1849 --- [  restartedMain] org.quartz.core.QuartzScheduler          : JobFactory set to: org.springframework.scheduling.quartz.SpringBeanJobFactory@56aedf13
2021-05-25 20:43:00.266  INFO 1849 --- [  restartedMain] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-05-25 20:43:01.895  INFO 1849 --- [  restartedMain] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'
2021-05-25 20:43:02.022  INFO 1849 --- [  restartedMain] o.s.b.d.a.OptionalLiveReloadServer       : LiveReload server is running on port 35729
2021-05-25 20:43:02.045  INFO 1849 --- [  restartedMain] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 14 endpoint(s) beneath base path '/actuator'
2021-05-25 20:43:02.546  INFO 1849 --- [  restartedMain] i.a.a.c.impl.AxonServerManagedChannel    : Requesting connection details from axon-server-grpc:8124
2021-05-25 20:43:02.878  INFO 1849 --- [  restartedMain] i.a.a.c.impl.AxonServerManagedChannel    : Successfully connected to axon-server-grpc:8124
2021-05-25 20:43:02.896  INFO 1849 --- [  restartedMain] i.a.a.connector.impl.ControlChannelImpl  : Connected instruction stream for context 'default'. Sending client identification
2021-05-25 20:43:02.910  INFO 1849 --- [  restartedMain] i.a.a.c.command.impl.CommandChannelImpl  : CommandChannel for context 'default' connected, 0 command handlers registered
2021-05-25 20:43:02.914  INFO 1849 --- [  restartedMain] i.a.a.c.command.impl.CommandChannelImpl  : Registered handler for command 'be.acerta.ce.calendar.api.RegisterAgreement' in context 'default'
2021-05-25 20:43:03.047  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 1s
2021-05-25 20:43:03.153  INFO 1849 --- [  restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2021-05-25 20:43:03.156  INFO 1849 --- [  restartedMain] o.s.s.quartz.SchedulerFactoryBean        : Starting Quartz Scheduler now
2021-05-25 20:43:03.178  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Freed 0 triggers from 'acquired' / 'blocked' state.
2021-05-25 20:43:03.182  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Recovering 0 jobs that were in-progress at the time of the last shut-down.
2021-05-25 20:43:03.182  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Recovery complete.
2021-05-25 20:43:03.184  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Removed 0 'complete' triggers.
2021-05-25 20:43:03.185  INFO 1849 --- [  restartedMain] o.s.s.quartz.LocalDataSourceJobStore     : Removed 0 stale fired job entries.
2021-05-25 20:43:03.188  INFO 1849 --- [  restartedMain] org.quartz.core.QuartzScheduler          : Scheduler MY_JOB_SCHEDULER_$_NON_CLUSTERED started.
2021-05-25 20:43:03.235  INFO 1849 --- [  restartedMain] b.a.ce.calendar.CalendarMgmtApplication  : Started CalendarMgmtApplication in 18.996 seconds (JVM running for 19.989)
2021-05-25 20:43:04.072  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 2s
2021-05-25 20:43:06.084  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 4s
2021-05-25 20:43:10.096  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 8s
2021-05-25 20:43:18.116  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 16s
2021-05-25 20:43:34.129  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 32s
2021-05-25 20:44:06.143  WARN 1849 --- [agaProcessor]-0] o.a.e.TrackingEventProcessor             : Fetch Segments for Processor 'AgreementLifecycleMgmtSagaProcessor' failed: Could not initialize segments. Some segments were already present.. Preparing for retry in 60s

Any ideas ?

The config is basically this:

@Configuration
//@AutoConfigureBefore(AxonAutoConfiguration.class)
public class AxonConfiguration {

    @Bean
    @ConfigurationProperties(prefix = "axon.acerta.tokenstore")
    public TokenStoreProperties tokenStoreProperties() {
        return new TokenStoreProperties();
    }

    @Bean
    @ConfigurationProperties(prefix = "axon.acerta.sagastore")
    public SagaStoreProperties sagaStoreProperties() {
        return new SagaStoreProperties();
    }


    @Bean
    //@ConditionalOnProperty(value = "axon.acerta.tokenstore.enabled", matchIfMissing = true)
    public TokenStore tokenStore(TokenSchema tokenSchema, DataSource dataSource, Serializer serializer) {
        ConnectionProvider connectionProvider = new DataSourceConnectionProvider(dataSource);
        return JdbcTokenStore.builder()
                .connectionProvider(connectionProvider)
                .claimTimeout(Duration.ofSeconds(10))
                .schema(tokenSchema)
                .nodeId(ManagementFactory.getRuntimeMXBean().getName())
                .serializer(serializer)
                .contentType(byte[].class)
                .build();
    }

    @Bean
    //@ConditionalOnProperty(value = "axon.acerta.sagastore.enabled")
    public SagaStore<Object> sagaStore(DataSource dataSource, SagaSqlSchema sagaSqlSchema, ConnectionProvider connectionProvider, Serializer serializer) {
        return JdbcSagaStore.builder()
                .connectionProvider(connectionProvider)
                .serializer(serializer)
                .sqlSchema(sagaSqlSchema)
                .dataSource(dataSource)
                .build();
    }

    @Bean
    public TokenSchema tokenSchema(TokenStoreProperties properties) {
        return TokenSchema.builder()
                .setTokenTable(String.format("%s.%s", properties.getSchema(), properties.getTableName()))
                .setOwnerColumn(properties.getOwnerColumnName())
                .setProcessorNameColumn(properties.getProcessornameColumnName())
                .setSegmentColumn(properties.getSegmentColumnName())
                .setTimestampColumn(properties.getTimestampColumnName())
                .setTokenColumn(properties.getTokenColumnName())
                .setTokenTypeColumn(properties.getTokentypeColumnName())
                .build();
    }

    @Bean
    public SagaSqlSchema sagaSqlSchema(SagaStoreProperties properties) {
        return new GenericSagaSqlSchema(new SagaSchema(String.format("%s.%s", properties.getSchema(), properties.getSagaTableName()),
                String.format("%s.%s", properties.getSchema(), properties.getAssociationTableName())));
    }

    @SuppressWarnings("SpringJavaInjectionPointsAutowiringInspection")
    @Bean
    public ScopeAwareProvider scopeAwareProvider(org.axonframework.config.Configuration configuration) {
        return new ConfigurationScopeAwareProvider(configuration);
    }

    @Bean
    public DeadlineManager deadlineManager(Scheduler scheduler,
                                           ScopeAwareProvider scopeAwareProvider,
                                           TransactionManager transactionManager,
                                           Serializer serializer) {
        return QuartzDeadlineManager.builder()
                .scheduler(scheduler)
                .scopeAwareProvider(scopeAwareProvider)
                .serializer(serializer)
                .transactionManager(transactionManager)
                .build();
    }
}

EDIT: in the mean time, I have added p6spy to inspect the queries:

 4ms | statement | connection 14| url jdbc:sqlserver://my-mssql-mssql-linux:1433;maxResultBuffer=-1;sendTemporalDataTypesAsStringForBulkCopy=true;delayLoadingLobs=true;useFmtOnly=false;useBulkCopyForBatchInsert=false;cancelQueryTimeout=-1;sslProtocol=TLS;jaasConfigurationName=SQLJDBCDriver;statementPoolingCacheSize=0;serverPreparedStatementDiscardThreshold=10;enablePrepareOnFirstPreparedStatementCall=false;fips=false;socketTimeout=0;authentication=NotSpecified;authenticationScheme=nativeAuthentication;xopenStates=false;sendTimeAsDatetime=true;trustStoreType=JKS;trustServerCertificate=false;TransparentNetworkIPResolution=true;serverNameAsACE=false;sendStringParametersAsUnicode=true;selectMethod=direct;responseBuffering=adaptive;queryTimeout=-1;packetSize=8000;multiSubnetFailover=false;loginTimeout=30;lockTimeout=-1;lastUpdateCount=true;encrypt=false;disableStatementPooling=true;databaseName=calendar;columnEncryptionSetting=Disabled;applicationName=Microsoft JDBC Driver for SQL Server;applicationIntent=readwrite;
SELECT processorName, segment, token, tokenType, timestamp, owner FROM ACERTA.TokenEntry WHERE processorName = ? AND segment = ? FOR UPDATE
SELECT processorName, segment, token, tokenType, timestamp, owner FROM ACERTA.TokenEntry WHERE processorName = 'AgreementLifecycleMgmtSagaProcessor' AND segment = 0 FOR UPDATE;

When I run the query myself, it returns an empty result.

Hello Christian,

Thank you for providing the example! We were able to reproduce the issue locally.
Once we turned on the debugging mode in logs, we got the whole exception:

org.axonframework.eventhandling.tokenstore.UnableToClaimTokenException: Could not initialize segments. Some segments were already present.
at org.axonframework.eventhandling.tokenstore.jdbc.JdbcTokenStore.lambda$initializeTokenSegments$4(JdbcTokenStore.java:147) ~[axon-messaging-4.5.jar:4.5]
at org.axonframework.common.jdbc.JdbcUtils.executeQuery(JdbcUtils.java:82) ~[axon-messaging-4.5.jar:4.5]
at org.axonframework.common.jdbc.JdbcUtils.executeQuery(JdbcUtils.java:55) ~[axon-messaging-4.5.jar:4.5]
at org.axonframework.eventhandling.tokenstore.jdbc.JdbcTokenStore.initializeTokenSegments(JdbcTokenStore.java:139) ~[axon-messaging-4.5.jar:4.5]
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.lambda$run$0(TrackingEventProcessor.java:1039) ~[axon-messaging-4.5.jar:4.5]
at org.axonframework.common.transaction.TransactionManager.fetchInTransaction(TransactionManager.java:70) ~[axon-messaging-4.5.jar:4.5]
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.run(TrackingEventProcessor.java:1036) ~[axon-messaging-4.5.jar:4.5]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Line 1: FOR UPDATE clause allowed only for DECLARE CURSOR.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1632) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:602) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:524) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7418) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3274) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:247) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:222) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:446) ~[mssql-jdbc-9.2.0.jre11.jar:na]
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) ~[HikariCP-3.4.5.jar:na]
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) ~[HikariCP-3.4.5.jar:na]
at org.axonframework.common.jdbc.JdbcUtils.executeQuery(JdbcUtils.java:80) ~[axon-messaging-4.5.jar:4.5]
… 6 common frames omitted

The exception is thrown by JdbcTokenStore.initializeTokenSegments (AxonFramework/JdbcTokenStore.java at 46861cf8436eef4e5c360176a9cd2a8aba92e09d · AxonFramework/AxonFramework · GitHub).

It looks like the MSSQL Server does not like the SELECT ... FOR UPDATE statement.

The JdbcTokenStore is implemented in such a way, that you can extend it in case databases do not support the SQL syntax that the default implementation uses. In this case, you’ll need to override a selectForUpdate method in the JdbcTokenStore (AxonFramework/JdbcTokenStore.java at 46861cf8436eef4e5c360176a9cd2a8aba92e09d · AxonFramework/AxonFramework · GitHub) or use the JPA implementation.

I hope this helps!

Best,
Ivan

Thank you for the quick answer.
The DB is MS SQL Server 2017.
So, shouldn’t we regard this as a Axon FW bug ?

It would be difficult to cover all SQL flavors by default in Axon Framework JdbcTokenStore. It is designed in a way you can extend the behaviour. We (at AxonIQ) are going to discuss if this is something we should be responsible for, within the framework. TBH, the priority of this is not high. I would recommend extending the JdbcTokenStore on your own, for now.

Ok, fair enough.
That being said, MS SQL Server is a quite common database system.

But in any case the error reported is quite misleading. It does not really reflect what is going on. And you swallow the underlying exception.

I used the JpaStore and all ok there for now. Thanks again.

1 Like

Hi @Christian_Bonami, you are right about swalling the error and that’s why we created an issue for that right away here.

About the Exception, to have that reproduced on our end and clear, we had to enable a DEBUG log on the app you shared and that’s was enough, thanks for that.
But keep in mind that having the DEBUG enabled when those ‘strange’ behaviours happens is a good advice IMO =)

And thanks for letting us know the JPATokenStore worked for you.