Unable to start AxonServer 4.4.1 with backup

Axon Server
#1

Hi there,

unfortunately we are unable to start our 4.2.4 AxonServer because of this issue: https://github.com/AxonIQ/axon-server-se/issues/105

So, I’m trying to update to Docker image axonserver:4.4.1. Without success so far.

First I found out that some of the old Environment parameters changed their names (AXONIQ_AXONSERVER_ACCESSCONTROL_ENABLED for example). Then I found out that we now have 3 different folders for data, events and config.

In 4.2.4, all files were in the same directory. Can I simple have something like this now?

      - AXONIQ_AXONSERVER_EVENT_STORAGE=/opt/axonserver/data
      - AXONIQ_AXONSERVER_SNAPSHOT_STORAGE=/opt/axonserver/data
      - AXONIQ_AXONSERVER_CONTROLDB-PATH=/opt/axonserver/data

Also, is the documentation on DockerHub correct in that it is CONTROLDB-PATH and not CONTROLDB_PATH (underscore)?

AxonServer is starting, but it doesn’t seem to be able to find any of my snapshots. Whenever I do something on an Aggregate, it just starts going through all of the historic events.

Martin

#2

Martin,
let me start with the simplest question: “CONTROLDB-PATH” is indeed a typo and it should be an underscore. We’ll update the documentation on Docker Hub.

As for the upgrade in general, indeed we changed to generating the container image using Jib, and that changed where it is installed and the way you need to pass settings. Rather than “/opt/axonserver” it is now installed in the root. The snapshots and events are now in “/eventdata”, with the control-db in “/data”.

Please note that it is probably simplest to actually mount a volume to “/config” and place an “axonserver.properties” file in it. This is described in my blogpost on running AxonServer in Docker and Kubernetes, which you can find on our site at https://axoniq.io/blog-overview/running-axon-server-in-docker.

Hope this helps you further,
Bert Laverman

#3

Thanks for the quick reply.

So, I can put all the files in the same directory (Docker volume), right? I just don’t get why AxonServer doesn’t seem to be able to find its old snapshots and starts creating new ones?

#4

You can, but snapshots and events each have their own property, so you need to specify both. If you specify nothing, it will use “/eventdata” for either, as the new image has those values in a property file at “/axonserver.properties”.

The default settings in the current image are:

axoniq.axonserver.event.storage=/eventdata
axoniq.axonserver.snapshot.storage=/eventdata
axoniq.axonserver.controldb-path=/data
axoniq.axonserver.pid-file-location=/data
logging.file=/data/axonserver.log
logging.file.max-history=10
logging.file.max-size=10MB

Cheers,
Bert

#5

That’s what I was trying to do with these 2 environment variables. Is it correct?

#6

And that is the location the volume is mounted? I would guess the easier solution would be to simply mount it on “/eventdata”. I’m not sure Docker is able to create “/opt/axonserver/data”. Let me test that.

Bert

#7

It is loading the events propery, so I guess it works. Only snapshots seem to be a problem. Maybe the snapshot problem is related to this issue that I just posted: There suddenly is a second *.snapshots file which prevents the server from starting

There suddenly is a second *.snapshots file which prevents the server from starting
#8

Martin,
I just tested using:

docker run --name axonserver -p 8024:8024 -p 8124:8124 -v `pwd`/events:/opt/axonserver/data -v `pwd`/data:/data -v `pwd`/config:/config -e AXONIQ_AXONSERVER_EVENT_STORAGE=/opt/axonserver/data -e AXONIQ_AXONSERVER_SNAPSHOT_STORAGE=/opt/axonserver/data axoniq/axonserver:4.4.1

After running this it appears to work correctly.

Cheers,
Bert Laverman

#9

That’s also what I think. But then I cannot figure out why AxonServer doesn’t seem to be finding my snapshot files.

I know that from: We have a very large aggregate with ~100k events on it. When I trigger some new event one it I see on the logs that it doesn’t use the snapshot but instead goes over ALL events because I see upcaster entries in the logs (and also it takes a veeeeeery long time to process).

#10

Martin,
since this is no longer about starting/running AxonServer, please start a fresh thread for that, so we can keep this searchable for others.

Bert

#11

You’re absolutely right, see: AxonServer 4.4.1 doesn't seem to find old (4.2.4) snapshots