Configuring Initial Segments count for TEP/PSEP through Axon Framework

Axon Framework
,
#1

Hi,

we are using Axon Framework 4.5.8 and Axon Server SE 4.5.10 and try to tune our event processing. Currently we’re running a single instance of our application and configure a TEP as follows:

TrackingEventProcessorConfiguration tepConfig =
            TrackingEventProcessorConfiguration.forParallelProcessing(8)
    .andInitialSegmentsCount(8)
    .andBatchSize(2000)
    .andInitialTrackingToken(StreamableMessageSource::createHeadToken);

configurer.registerTrackingEventProcessor("my-processor", c -> (StreamableMessageSource) c.eventBus(), s -> tepConfig);

respectively for testing purposes a PSEP as follows

Function<String, ScheduledExecutorService> coordinatorExecutorBuilder =
    name -> Executors.newScheduledThreadPool(1, new AxonThreadFactory("Coordinator - " + name));

Function<String, ScheduledExecutorService> workerExecutorBuilder =
    name -> Executors.newScheduledThreadPool(8, new AxonThreadFactory("Worker - " + name));

EventProcessingConfigurer.PooledStreamingProcessorConfiguration psepConfig =
            (config, builder) -> builder.coordinatorExecutor(coordinatorExecutorBuilder)
    .workerExecutor(workerExecutorBuilder)
    .initialSegmentCount(8)
    .batchSize(300)
    .initialToken(StreamableMessageSource::createHeadToken);

configurer
    .registerPooledStreamingEventProcessor(AclManagementEventHandler.PROCESSOR)
    .registerPooledStreamingEventProcessorConfiguration(AclManagementEventHandler.PROCESSOR, psepConfig);

We are running our application against an axon server instance that is initialized with an existing dump of a control database and an existing dump of events/index files.

Regardles of the initialSegmentCount we set for the TEP or PSEP through the Axon framework configuration, Axon Server always uses 3 segments for the TEP and PSEP and we need to manually split segments through the axon server dashboard to get to the desired number of segments.

I’ve also checked our application properties and we do not override the number of segments anywhere, so I’m wondering
a) why Axon Server is not taking the number of initial segments configured for the processor on the framework side and
b) where the number of 3 segments comes from

Does Axon server somehow remember the number of segments used for a processor previously by it’s name somehow in the control database? That might explain it, because we are using an existing dump to initialize axon server as said before and the processor might previously be run with 3 segments. But still then i would question if that behaviour is desired, because I would expect the number of segments for the processor to be as configured through the framework.

Any inputs on this are appreciated,

Thanks and best Regards,
Jakob

#2

Hey @Jakob_Hatzl,

Axon Server does not store any information on the segment count for event processors.
That’s all the Framework applications.

From the snippets you’ve shared, I cannot immediately deduce where the count of three segments comes from.
What I do know is that it isn’t an Axon Framework default.
By the way, the default segment count for a TrackingEventProcessor is 1, whereas the default for a PooledStreamingEventProcessor is 16.

What’s good to note is that the segments are taken from the TokenStore.
If the TokenStore already contains segments for a given processor name, the processors will not adjust the amount.
This is why the configuration option is called initialSegmentCount and initialToken.

Having said that, I’d take a look at the configured database for the TokenStore.
I am guessing it still contains tokens from a previous run.

#3

Hi Steven,

thanks for your response. Didn’t think of the token store and there are indeed 3 segments existing for the processor in question in the dump we are using to init the database providing the token store. That explains the behaviour. Also the initial prefix to the method should have told that, now that I know it.

So that would leave axon server dashboard or using the API of StreamingEventProcessor at runtime in code to split segments until we get the desired number of segments.

1 Like
#4

Awesome, I am happy to hear that solved the problem for you, @Jakob_Hatzl!