Event Storage disk usage and hidden usage costs

Imported from Google Groups
#1

Hi everybody,

At the moment we’re considering Axon as a base for our next application we will develop in near future.
The run-time costs is one of factors, that had concerned some of our devs and I’m trying to do some estimates about it.

The problem is that it is very hard to find some general performance, especially disk usage information - or even better deep analysis - of AXON/CQRS impact of needed space and disk ops.

Of course I know it is impossible to give some strict relationship between event count and disk space occupied by this, but I would like to have some point of reference.

To the point:
Could you give me some overview on event store disk usage, the best in context of store type, event count and so on (answers like “1000000 event takes about 1GB on our MySQL store” or “1000 events does 2000 disk ops on our AWS machines” would be sufficient for me).

Thanks in advice, Chris

#2

Hi Chris,

We happen to be running some event store benchmarking right now at AxonIQ, so I can share some data from that. Of course, there’s a huge number of variables here, so the real answer is “it depends”. The numbers below should be seen as an example.

We’re currently trying to store a large number of events on a MySQL database.

  • Total size on disk (size of /var/lib/mysql) is currently 260G.

  • Total number of events stored is 180M - so average disk usage per event is 1.444 bytes.

  • The main data included in our events on the application level is on average 200 bytes.

  • Throughput right now is about 950 events/second, with on average 5 events in a transaction.

  • This takes 1250 write IOPS (in the Google cloud, on a n1-standard-8 machine, 1TB HDD for the data)
    One thing to keep in mind when selecting an event store is the total number of events you are planning to store. Traditional relational databases may become a bottleneck if storing very large numbers of events.

The reason we’re running these benchmarks now, is that we’ll be launching a dedicated event store shortly - this one is optimized for the task and behaves better when storing 100M to billions of events. If you’re interested in learning more about this, or just discuss your plans in general, please contact me.