I have attended to the 2019 AxonIQ conference and there I got the suggestion that Axon Server might not perform well on Kubernetes on production environment, so most companies use Axon Server as an infrastructure in a clustered manner. I still see some posts from the past in this portal that suggests the same thing.
However, I see that there’s documentation about Kubernetes and Docker in the new Axon Server documentation(https://docs.axoniq.io/reference-guide/axon-server/installation/docker-k8s). Does this mean that, from now on, AxonIQ recommends and supports the usage of Axon Server with Kubernetes and Docker?
For a helm chart there is some work here (but only valid with SE version. I changed few tings to support EE):
The helm for EE will be updated in a near future (just make it work this week based on the SE one).
The scale up to 3 nodes with EE version was not working as expected based on documentation and axoniq github. Hostname resolution was wrong because the pod_name.service_name.namespace.srv.cluster.local is not an existing record.
I have not yet discussed with Axoniq about that to share why it is working for them and not for us.
We just evaluated the SE version on a small load so I will share when we will stress test all of that.
Anil,
thank you for your interest in AxonServer on Kubernetes. You may be happy to know that actually several of our customers deploy AxonServer to Kubernetes without troubles. Kubernetes has always been a supported platform, but, and that is probably what was referred to, there are some points you have to take into account. Indeed I went into that subject in the “Running AxonServer” blog series, so you might want to read parts 2 and especially 3 - resp. about running in Docker and Kubernetes, vs running in a VM - about the reasons why you might want to choose the one or the other. Note that the choice is most relevant for clusters of AxonServer EE if you want to create a multi-regional setup. For all the major cloud providers this is still not possible with their out-of-the-box Kubernetes solutions. Also, with release 4.4 you can use multi-tier storage where you have to (want to) differentiate in the type of storage between nodes, which reduces the advantages Kubernetes offers for ease of deployment.
So, Kubernetes is for those situations where your deployment patterns are (relatively) simple, and there it is unbeatable. If you have more complex requirements concerning your infrastructure, then VM’s will let you do everything you might want, at the cost of more preparation time.
So let me know if you have any specific questions about Kubernetes deployments, and we’ll help you along!
Thank you for the nice and informative responses. I think this gives some overview of the pros and cons of the available choices. We’ll contact you if we have more questions about the deployments, but the articles should at least help us to begin that journey.
I have only one follow-up after this: Is there a plan for an official Helm chart for Axon Server EE and will you put that on the Helm Hub?
Mickey,
can you please share your work so I can take a look? I have manual deployments to Kubernetes in the running-axon-server repo working, so I am intrigued where you ran into DNS troubles. Please do note that in the case of scaling, Kubernetes will not create the DNS record until the pod is reported as ready! This can create some confusing exceptions in the logs.