Situation:
We have an existing integration context of about 50 million events.
Our new application has an event listener that will start processing those 50 mil events to create and update aggregates in it’s own context.
The simple implementation take about 3 days to process those 50 mil events.
@Eventhandler
public void on(IntegrationEvent e) {
commandGateway.sendAndWait(toLocalCommand(e));
}
I’ve implemented the same solutions as presented in CQRS replay performance tuning (8 segments, batch size 500, use unitOfWork to combine multiple events into one command)
To send the commands I have 2 options and each comes with it’s own problems:
- commandGateway.send:
This processes the events really fast but we are getting a lot of command timeouts in our log.
Even an hour after stopping the eventprocessor using axonserver dashboard, we still see command beeing executed.
I think a timeout on the sending side does not mean the command will not be executed?
I suspect axonserver is using some kind of queue to store the commands?
Is it possible we send too many commands for axonserver to process? Or will axonserver try to execute all commands it has received?
- commandGateway.sendAndWait:
Processing of events is slower and segments are getting claimed by other threads because processing of one batch takes too long. (so we end up with temporary duplicate segment claims)
This means we have to tweak batch size or maybe claimTimeout of the tokenstore?
Are there suggestions for perfromance tuning this use case?
Thanks,
M