I am trying to replicate the experiment performed in the following document (https://www.digitalfrontiers.de/wp-content/uploads/2022/04/Digital-Frontiers_Axon-Server-Benchmarks-1.pdf).
I am not sure how the “events/sec” (event rate) is measured as we keep publishing the events in the event store. What tool or method is employed to measure the event rate so that I can get the similar plots for measuring the throughput performance of the Axon Server?
Thanks
Hi Aman,
I did that benchmark back then when working for DigitalFrontiers, so I am happy to provide you with some insights.
The experiments were run on a cloud-hosted server at Hetzner, one of their CPX31 tier.
The test application used for Slides 9-14 did use a single thread to produce events. It did that by issuing commands creating anew aggregate. I measured the aggregate creation rate to indirectly measure the event rate. Note that the application did not interleave commands and always waited for acknowledgement before sending the next one.
On later slides, I increased the number of parallel threads for both, postgres and axon but kept the requirement of waiting for ACKs.
Note that in the presentation I messed up the scaling of the X axis at some places: The tests were run with up to 100 Million events in the store to determine if performance decreases over time or with an increasing amount of events in the event store (as you can see with mongo and postgres). The test on slide 12 was run with a billion events, then my cheap server was running out of space.
One note: I chose one of the worst-possible workload-models here (since i wanted to challenge the claims axoniq made back then) and mixed writes (create events) and read them (source them for the aggregate). If you want to increase the total throughput, you can publish events directly without all the other stuff. With a bit of microbatching I was able to reach 200K events per second on my dev machine.
I planned to write a blogpost on how to reproduce the benchmarks and publish the code. It is still on my agenda but I did not yet get around to do it.
Kind regards,
Marco
Thanks Macro for the detailed description.
We are also trying to replicate the following experiment and your answer would help us a lot.
It will really help the community once that blog will be out 