How to handle ever growing disk usage

Hi there,

when you have an Axon Server running for a long time, obviously the disk usage will grow and grow. Is there any mechanism to “shrink” the disk usage at some point? Like, only store a snapshot of events before date X, and delete older events. Or something similar?

Or does one just have to keep buying more and more disk space?

Martin

Hi Martin,

the main idea is that you will keep the events for future reference. Obviously, there are situations where it is fine to remove old data, but I believe that is a choice that should not be taken lightly. Once the data is gone… its… gone.

One thing that might help is a new feature that we introduced in Axon Server 4.4: Secondary Nodes. These are nodes that (just like Primary Nodes) replicate the data, but they allow the Primary Nodes to evict data from their disks, knowing that the data resides on (all of) the Secondary Nodes.

While Primary Nodes would need 3 instances to get a HA cluster, you only need 2 Secondary Nodes for HA. That’s already one replica of the data less.

Lastly, there are features on the roadmap to reduce the storage cost of event files, most likely at the cost of increased access time. One example is compression, but another is to allow moving segment files to different storage locations, such as S3.

Hope this helps.

2 Likes

Hey!

Nice approaches. I like the one where we can store older files in other folders (like, mapped folders from cloud drives) best :+1:

Do you have any rough idea about when we might expect any of the solutions to be available?