How to delete old snapshots with AxonServerEventStore

Hi everyone.
We are using the AxonServerEventStore to store our Events and Snapshots. While testing I assumed that when a new snapshot for an Aggregate was created, the old one would be deleted. This was not the case. So I did a bit of research and found out that this is supposed to be the default behaviour of JDBC and JPA-EventStores but I could not find anything about the AxonServerEventStore

Then I found a method deleteSnapshot but it was only available in the JDBC and JPA-EventStoreEngines.

Is it even possible to delete older snapshots using the AxonServerEventStore and AxonIQEventStorageEngine?

We are using Axon Server 4.5.6 and the framework version 4.5.
That’s our config:

  @Bean
  public SpringAggregateSnapshotter snapshotter(
      final ParameterResolverFactory parameterResolverFactory,
      final EventStore eventStore,
      final TransactionManager transactionManager) {
    final var executor = Executors.newFixedThreadPool(10);
    return SpringAggregateSnapshotter.builder()
        .eventStore(eventStore)
        .parameterResolverFactory(parameterResolverFactory)
        .executor(executor)
        .transactionManager(transactionManager)
        .build();
  }

  @Bean(name = "projectSnapshotTriggerDefinition")
  public SnapshotTriggerDefinition projectSnapshotTriggerDefinition(final Snapshotter snapshotter) {
    return new EventCountSnapshotTriggerDefinition(snapshotter, 150);
  }

Hi @danielvdwal,

Unfortunately, it is not able to delete snapshots when using Axon Server yet.
But that is not a problem at all as they will be stored on your Event Store.

A thing to mention is the usage of the @Revision annotation on Aggregates which can be used to filter out old snapshots when you have to change the structure of your Aggregate.

KR,

Thanks for your reply. I am aware about the @Revision Annotation and used it in some of my tests as well and it worked great. But the problem of keeping the old snapshots still remains. The thing is, one particular aggregate can grow alot and after some time (for instance having the first snapshot on the 150th event), the payload is approx. 500kb large. If we cannot delete snapshots and the aggregate is able to grow even larger we might run out of storage.
In the worst case the aggegate grows linear, let’s say the 300th-Event-Snapshot has a size of 1MB, 450th-Event 1,5MB and so on. after about 3000 events one single snapshot has a size of 10MB and while keeping the old events, we would have wasted about 90 to 100MB for all 9 old events.

Is it possible to remove the snapshot by hand? I could not find a Rest-Endpoint to delete one.

Hi @danielvdwal, although I understand your concerns, I have to say that something might be off with this Aggregate.

Having an ever growing Snapshot is not that common - it might indicate you have some sort of Collection inside it, which is serialized and saved on your Snapshot.
You can have potential problems in the future with that and I would look if you can somehow re-design it.
I am saying that because gRPC has a max-message-size of 4MB and as soon as you cross that limit, you will start having problems. Of course you can change this size and make it bigger but I would rather look into why you are storing a Collection instead of just bumping the size.

KR,

Hi @lfgcampos
Yes you are right, we are now redesigning that aggregate. We are storing to much information inside sub-objects within a collection as you already figured out.

But is there no way to delete old snapshots? We found the snapshot file but it seems to allocate around 256MB and stores all snapshots in that one file even though our snapshots combined will never sum up to that high number at the moment. What happens if we simply delete this file? Will Axon create just one new snapshot per aggregate the next time the aggregates are loaded? Or will the AxonEventStore freak out, because we removed the snapshot file?

Kind regards, Daniel

Hi @danielvdwal,

Good to hear as I believe a re-design is the way to go!

For your question about deleting it, it depends =)
If you are using Axon Server Standard Edition, if you stop it, delete the files and the indexes for the snapshots and restart it later should be fine - the extensions you are looking for are .snapshots, .sindex, .snindex, .sbloom and .sxref. You should be careful to delete those files for snapshots only and not for events!
If you are using Axon Server Enterprise Edition, because of the clustering feature, it gets way more complicated.

For the file size, Axon Server will always make a 256MB file for Events and another for Snapshots. It is just a way to ‘reserve’ this amount of space on disk and it will fill it in with events and snapshots. When the file is filled, it will create another 256MB file and so on.
But it does not mean your Event/Snapshot has 256MB.

Those files are what we call segments!
You can also change the size of it using the axoniq.axonserver.snapshot.segment-size property.

Hope that answers your question.

KR,