I’d like to put all of the Axon guts (events, sagas, tokens, quartz, etc) into a separate database (either postgres or mongo) than the views (postgres). I’ve seen this come up a few times but I can’t find a concrete example! Any guidance would be appreciated.
Hi,
i did a Showcase on how to use axon with microprofile on quarkus.
If you are looking for spring setup, it might not be helpful, since axon configuration is done using the config api.
Command- and Query-Side use separate datasources and can be configured to different databases.
The Token table should be on Query-Side where your projection is. This is easier than xa transactions.
The showcase also contains flyway setups for h2 and Postgres.
Kind regards. Johnny.
Thank you for the reply Johnny. I am using Spring Boot but your example is very helpful.
The configuration I’ve come up with for Axon 4 on Spring Boot 2.2 is rather obtuse, but it works. I assume this is because I have to bootstrap everything manually rather than using existing auto configuration. I started with the linked GitHub project, and then added pieces from FlywayAutoConfiguration. All hibernate properties must be placed in events.jpa.properties.hibernate rather than events.jpa.hibernate.
@Configuration
class EventStoreJpaConfig {
/************************************************************************
* Start with the basic JDBC datasource
************************************************************************/
@Bean
@Qualifier("events")
@ConfigurationProperties(prefix = "events.datasource")
fun eventsDataSource(): DataSource {
return DataSourceBuilder.create().build()
}
/************************************************************************
* Using Flyway to do the required schema creation/updates
************************************************************************/
@Configuration
@ConditionalOnProperty(prefix = "events.flyway", name = ["enabled"], matchIfMissing = true)
@Import(
FlywayConfiguration.FlywayEntityManagerFactoryDependsOnPostProcessor::class,
FlywayConfiguration.FlywayJdbcOperationsDependsOnPostProcessor::class,
FlywayConfiguration.FlywayNamedParameterJdbcOperationsDependencyConfiguration::class,
FlywayConfiguration.FlywayMigrationInitializerEntityManagerFactoryDependsOnPostProcessor::class,
FlywayConfiguration.FlywayMigrationInitializerJdbcOperationsDependsOnPostProcessor::class,
FlywayConfiguration.FlywayMigrationInitializerNamedParameterJdbcOperationsDependsOnPostProcessor::class
)
class FlywayConfiguration {
@Bean
@Qualifier("events")
@ConfigurationProperties(prefix = "events.flyway")
fun eventsFlywayProperties(): FlywayProperties {
return FlywayProperties()
}
@Bean
@Qualifier("events")
fun eventsFlyway(
@Qualifier("events") dataSource: DataSource,
@Qualifier("events") flywayProperties: FlywayProperties
): Flyway {
return Flyway.configure()
.baselineOnMigrate(flywayProperties.isBaselineOnMigrate)
.dataSource(dataSource)
.locations(*flywayProperties.locations.toTypedArray())
.schemas(*flywayProperties.schemas.toTypedArray())
.load()
}
@Bean
fun eventsFlywayMigrationInitializer(@Qualifier("events") flyway: Flyway): FlywayMigrationInitializer {
return FlywayMigrationInitializer(flyway)
}
@Qualifier("events")
@ConditionalOnClass(LocalContainerEntityManagerFactoryBean::class)
@ConditionalOnBean(AbstractEntityManagerFactoryBean::class)
class FlywayEntityManagerFactoryDependsOnPostProcessor :
EntityManagerFactoryDependsOnPostProcessor(Flyway::class.java)
@Qualifier("events")
@ConditionalOnClass(JdbcOperations::class)
@ConditionalOnBean(JdbcOperations::class)
class FlywayJdbcOperationsDependsOnPostProcessor :
JdbcOperationsDependsOnPostProcessor(Flyway::class.java)
@Qualifier("events")
@ConditionalOnClass(NamedParameterJdbcOperations::class)
@ConditionalOnBean(NamedParameterJdbcOperations::class)
class FlywayNamedParameterJdbcOperationsDependencyConfiguration :
NamedParameterJdbcOperationsDependsOnPostProcessor(Flyway::class.java)
@Qualifier("events")
@ConditionalOnClass(LocalContainerEntityManagerFactoryBean::class)
@ConditionalOnBean(AbstractEntityManagerFactoryBean::class)
class FlywayMigrationInitializerEntityManagerFactoryDependsOnPostProcessor :
EntityManagerFactoryDependsOnPostProcessor(FlywayMigrationInitializer::class.java)
@Qualifier("events")
@ConditionalOnClass(JdbcOperations::class)
@ConditionalOnBean(JdbcOperations::class)
class FlywayMigrationInitializerJdbcOperationsDependsOnPostProcessor :
JdbcOperationsDependsOnPostProcessor(FlywayMigrationInitializer::class.java)
@Qualifier("events")
@ConditionalOnClass(NamedParameterJdbcOperations::class)
@ConditionalOnBean(NamedParameterJdbcOperations::class)
class FlywayMigrationInitializerNamedParameterJdbcOperationsDependsOnPostProcessor :
NamedParameterJdbcOperationsDependsOnPostProcessor(FlywayMigrationInitializer::class.java)
}
/************************************************************************
* Configuring JPA
************************************************************************/
@Bean
@Qualifier("events")
@ConfigurationProperties(prefix = "events.jpa")
fun eventsJpaProperties(): JpaProperties {
return JpaProperties()
}
@Bean
@Qualifier("events")
fun eventsEntityManagerFactory(
builder: EntityManagerFactoryBuilder,
@Qualifier("events") dataSource: DataSource,
@Qualifier("events") jpaProperties: JpaProperties
): LocalContainerEntityManagerFactoryBean {
return builder
.dataSource(dataSource)
.properties(jpaProperties.properties)
.packages(
"org.axonframework.eventsourcing.eventstore.jpa",
"org.axonframework.modelling.saga.repository.jpa"
)
.persistenceUnit("events")
.build()
}
@Bean
@Qualifier("events")
fun eventsPlatformTransactionManager(
@Qualifier("events") entityManagerFactory: EntityManagerFactory
): PlatformTransactionManager {
return JpaTransactionManager(entityManagerFactory)
}
@Bean
@Qualifier("events")
fun eventsSharedEntityManager(@Qualifier("events") entityManagerFactory: EntityManagerFactory): EntityManager {
return SharedEntityManagerCreator.createSharedEntityManager(entityManagerFactory)
}
/************************************************************************
* Axon Framework specific things
************************************************************************/
@Bean
@Qualifier("events")
fun eventsEntityManagerProvider(@Qualifier("events") entityManager: EntityManager): EntityManagerProvider {
return SimpleEntityManagerProvider(entityManager)
}
@Bean
@Qualifier("events")
fun eventsTransactionManager(@Qualifier("events") transactionManager: PlatformTransactionManager): TransactionManager {
return SpringTransactionManager(transactionManager)
}
@Bean
@Qualifier("events")
@Throws(SQLException::class)
fun eventsDataSourcePER(@Qualifier("events") dataSource: DataSource): PersistenceExceptionResolver {
return SQLErrorCodesResolver(dataSource)
}
@Bean
@Primary
fun eventStorageEngine(
eventSerializer: Serializer,
snapshotSerializer: Serializer,
@Qualifier("events") dataSource: DataSource,
@Qualifier("events") entityManagerProvider: EntityManagerProvider,
@Qualifier("events") persistenceExceptionResolver: PersistenceExceptionResolver,
@Qualifier("events") transactionManager: TransactionManager
): EventStorageEngine {
return JpaEventStorageEngine.builder()
.eventSerializer(eventSerializer)
.snapshotSerializer(snapshotSerializer)
.dataSource(dataSource)
.entityManagerProvider(entityManagerProvider)
.persistenceExceptionResolver(persistenceExceptionResolver)
.transactionManager(transactionManager)
.build()
}
@Bean
@Primary
fun sagaStore(
serializer: Serializer,
@Qualifier("events") entityManagerProvider: EntityManagerProvider?
): JpaSagaStore {
return JpaSagaStore.builder()
.entityManagerProvider(entityManagerProvider)
.serializer(serializer)
.build()
}
@Bean
@Primary
fun commandBus(
axonConfiguration: AxonConfiguration,
duplicateCommandHandlerResolver: DuplicateCommandHandlerResolver,
@Qualifier("events") txManager: TransactionManager
): SimpleCommandBus {
val commandBus = SimpleCommandBus.builder()
.transactionManager(txManager)
.duplicateCommandHandlerResolver(duplicateCommandHandlerResolver)
.messageMonitor(axonConfiguration.messageMonitor(CommandBus::class.java, "commandBus"))
.build()
commandBus.registerHandlerInterceptor(
CorrelationDataInterceptor(axonConfiguration.correlationDataProviders())
)
return commandBus
}
}
Hi,
getting multiple datasources to work in Spring Boot does indeed require manual configuration, afaik. Nothing Axon can do about that.
We do indeed recommend to store Tracking Tokens in the same database as your projections. With regards to separating ‘Axon guts’ from the rest, you provably really only want to separate the Event Store from the rest. Sagas should live in the database ‘owned’ by the service that runs the Saga. Events could be considered ‘shared goods’.
In fact, if you’d use AxonServer, that’s exactly what you’d get. Plus the location transparency through (distributed) messaging.
Cheers,
Allard
Perfect. I’d like to use AxonServer as long as the guts can be stored in an external database. I’ve posted about this previously, but haven’t circled back to fully implement.
Hi Joel,
In the Enterprise edition, you can, but we don’t recommend it, actually. It will have a severe impact on your performance in the long run, as relational databases aren’t suited for storage of large event streams.
We could talk about the details of it. Depending on the exact “why” of this desire/requirement, there are either workarounds that don’t suffer the performance loss (and still get the data in the relational database), or we’ll just help you configure the database as the backing storage.
Cheers,
20 Ocak 2020 Pazartesi 22:19:25 UTC+3 tarihinde Allard Buijze yazdı:
Hi,
getting multiple datasources to work in Spring Boot does indeed require manual configuration, afaik. Nothing Axon can do about that.
We do indeed recommend to store Tracking Tokens in the same database as your projections. With regards to separating ‘Axon guts’ from the rest, you provably really only want to separate the Event Store from the rest. Sagas should live in the database ‘owned’ by the service that runs the Saga. Events could be considered ‘shared goods’.
In fact, if you’d use AxonServer, that’s exactly what you’d get. Plus the location transparency through (distributed) messaging.Cheers,
Allard
What i understood from the answer is that if “you provably really only want to separate the Event Store from the rest” is what Joel meant then Joel does not need external database (as it is well explained in GOTO 2019 Event store in Axon How does it work) providing that axon server is used. Read side (CQRS) postgre is ok.
But, i have a question regarding saga store. In this specific case (write DB is Axon event store, read DB is postgre) which one is the saga store?
Thanks.
I’ve stored sagas on the event side. The delineation in my mind is “things that are mission critical” and “things I can drop on a whim”.
Thanks for referencing the video, I will watch that today. I clearly have a lot to learn about Axon Server, however the primary benefit of postgres in my mind is that it’s a known quantity and battle tested over multiple decades. DevOps folks know how to snapshot and secure a postgres server (or cluster), but perhaps not the case w/ Axon Server. I’m happy to be proven wrong about this! I couldn’t care less about the relational aspect for the event store, it’s the environment.