Issue
AxonServer not starting.
Error logs (DEBUG)
Bellow is the error message I got
axonserver-1 | _ ____
axonserver-1 | / \ __ _____ _ __ / ___| ___ _ ____ _____ _ __
axonserver-1 | / _ \ \ \/ / _ \| '_ \\___ \ / _ \ '__\ \ / / _ \ '__|
axonserver-1 | / ___ \ > < (_) | | | |___) | __/ | \ V / __/ |
axonserver-1 | /_/ \_\/_/\_\___/|_| |_|____/ \___|_| \_/ \___|_|
axonserver-1 | 2024.2.1 Powered by AxonIQ
axonserver-1 |
axonserver-1 | 2025-01-16T14:55:37.883Z INFO 1 --- [Axon Server] [ main] io.axoniq.axonserver.AxonServer : Starting AxonServer using Java 17.0.13 with PID 1 (/axonserver/axonserver.jar started by root in /axonserver)
axonserver-1 | 2025-01-16T14:55:37.887Z INFO 1 --- [Axon Server] [ main] io.axoniq.axonserver.AxonServer : No active profile set, falling back to 1 default profile: "default"
axonserver-1 | 2025-01-16T14:55:43.357Z INFO 1 --- [Axon Server] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port 443 (https)
axonserver-1 | 2025-01-16T14:55:43.621Z INFO 1 --- [Axon Server] [ main] A.i.a.a.c.MessagingPlatformConfiguration : Configuration initialized with SSL ENABLED and access control ENABLED.
axonserver-1 | 2025-01-16T14:55:49.606Z INFO 1 --- [Axon Server] [ main] io.axoniq.axonserver.AxonServer : Axon Server version 2024.2.1
axonserver-1 | 2025-01-16T14:55:50.136Z DEBUG 1 --- [Axon Server] [ main] io.axoniq.axonserver.ClusterTagsCache : This node is configured with tag names: []
axonserver-1 | 2025-01-16T14:55:56.548Z DEBUG 1 --- [Axon Server] [ main] i.a.a.rest.WebSecurityConfiguration : Using default security configurer.
axonserver-1 | 2025-01-16T14:55:56.549Z DEBUG 1 --- [Axon Server] [ main] i.a.a.rest.WebSecurityConfigurer : Configuring Web Security.
axonserver-1 | 2025-01-16T14:55:56.551Z DEBUG 1 --- [Axon Server] [ main] i.a.a.rest.WebSecurityConfigurer : Access control is ENABLED. Setting up filters and matchers.
axonserver-1 | 2025-01-16T14:55:57.911Z WARN 1 --- [Axon Server] [ main] i.m.core.instrument.MeterRegistry : This Gauge has been already registered (MeterId{name='disk.free', tags=[tag(axonserver=axon),tag(path=/)]}), the Gauge registration will be ignored. Note that subsequent logs will be logged at debug level.
axonserver-1 | 2025-01-16T14:55:57.935Z DEBUG 1 --- [Axon Server] [ main] i.a.a.plugin.SystemPackagesProvider : Adding exports from io.axoniq.axonserver-plugin-api to system packages path
axonserver-1 | 2025-01-16T14:55:57.937Z DEBUG 1 --- [Axon Server] [ main] i.a.a.plugin.SystemPackagesProvider : Adding exports from org.osgi.service.log to system packages path
axonserver-1 | 2025-01-16T14:55:57.939Z DEBUG 1 --- [Axon Server] [ main] i.a.a.plugin.SystemPackagesProvider : Adding exports from io.axoniq.axonserver-plugin-api to system packages path
axonserver-1 | 2025-01-16T14:55:57.940Z DEBUG 1 --- [Axon Server] [ main] i.a.a.plugin.SystemPackagesProvider : Adding exports from org.osgi.service.log to system packages path
axonserver-1 | 2025-01-16T14:55:57.940Z INFO 1 --- [Axon Server] [ main] i.a.axonserver.plugin.OsgiController : System packages io.axoniq.axonserver.plugin;version="4.8.1",io.axoniq.axonserver.plugin.hook;uses:="io.axoniq.axonserver.grpc.event,io.axoniq.axonserver.plugin";version="4.8.1",io.axoniq.axonserver.plugin.interceptor;uses:="io.axoniq.axonserver.grpc.command,io.axoniq.axonserver.grpc.event,io.axoniq.axonserver.grpc.query,io.axoniq.axonserver.plugin";version="4.8.1",io.axoniq.axonserver.grpc;uses:="com.google.protobuf";version="4.8.1",io.axoniq.axonserver.grpc.admin;uses:="com.google.protobuf,io.axoniq.axonserver.grpc,io.axoniq.axonserver.grpc.control";version="4.8.1",io.axoniq.axonserver.grpc.command;uses:="com.google.protobuf,io.axoniq.axonserver.grpc";version="4.8.1",io.axoniq.axonserver.grpc.control;uses:="com.google.protobuf,io.axoniq.axonserver.grpc";version="4.8.1",io.axoniq.axonserver.grpc.event;uses:="com.google.protobuf,io.axoniq.axonserver.grpc";version="4.8.1",io.axoniq.axonserver.grpc.query;uses:="com.google.protobuf,io.axoniq.axonserver.grpc";version="4.8.1",io.axoniq.axonserver.grpc.streams;uses:="com.google.protobuf,io.axoniq.axonserver.grpc.event";version="4.8.1",com.google.protobuf;version="3.25.1",com.google.protobuf.compiler;version="3.25.1";uses:="com.google.protobuf",org.osgi.service.log;version="1.5";uses:="org.osgi.framework",org.osgi.service.log.admin;version="1.0";uses:="org.osgi.service.log"
axonserver-1 | 2025-01-16T14:55:58.222Z WARN 1 --- [Axon Server] [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'MessagingClusterServer'
axonserver-1 | 2025-01-16T14:55:58.565Z ERROR 1 --- [Axon Server] [ main] o.s.boot.SpringApplication : Application run failed
axonserver-1 |
axonserver-1 | org.springframework.context.ApplicationContextException: Failed to start bean 'MessagingClusterServer'
axonserver-1 | at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:288) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:472) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
axonserver-1 | at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:257) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:202) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:990) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:628) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.3.7.jar!/:3.3.7]
axonserver-1 | at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.3.7.jar!/:3.3.7]
axonserver-1 | at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.3.7.jar!/:3.3.7]
axonserver-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) ~[spring-boot-3.3.7.jar!/:3.3.7]
axonserver-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1363) ~[spring-boot-3.3.7.jar!/:3.3.7]
axonserver-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1352) ~[spring-boot-3.3.7.jar!/:3.3.7]
axonserver-1 | at io.axoniq.axonserver.AxonServer.main(t:135) ~[!/:na]
axonserver-1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
axonserver-1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
axonserver-1 | at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
axonserver-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:569) ~[na:na]
axonserver-1 | at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:102) ~[axonserver.jar:na]
axonserver-1 | at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:64) ~[axonserver.jar:na]
axonserver-1 | at org.springframework.boot.loader.launch.PropertiesLauncher.main(PropertiesLauncher.java:580) ~[axonserver.jar:na]
axonserver-1 | Caused by: io.axoniq.axonserver.exception.FailedToStartException: [AXONIQ-0001] Starting Axon Server Cluster Server failed
axonserver-1 | at io.axoniq.axonserver.enterprise.cluster.internal.MessagingClusterServer.start(pca:180) ~[!/:na]
axonserver-1 | at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:285) ~[spring-context-6.1.16.jar!/:6.1.16]
axonserver-1 | ... 20 common frames omitted
axonserver-1 | Caused by: java.security.spec.InvalidKeySpecException: Neither RSA nor EC worked
axonserver-1 | at io.grpc.util.CertificateUtils.getPrivateKey(CertificateUtils.java:90) ~[grpc-util-1.68.2.jar!/:1.68.2]
axonserver-1 | at io.grpc.util.AdvancedTlsX509KeyManager.readAndUpdate(AdvancedTlsX509KeyManager.java:288) ~[grpc-util-1.68.2.jar!/:1.68.2]
axonserver-1 | at io.grpc.util.AdvancedTlsX509KeyManager.updateIdentityCredentials(AdvancedTlsX509KeyManager.java:141) ~[grpc-util-1.68.2.jar!/:1.68.2]
axonserver-1 | at io.grpc.util.AdvancedTlsX509KeyManager.updateIdentityCredentialsFromFile(AdvancedTlsX509KeyManager.java:214) ~[grpc-util-1.68.2.jar!/:1.68.2]
axonserver-1 | at io.axoniq.axonserver.enterprise.cluster.internal.MessagingClusterServer.start(pca:106) ~[!/:na]
axonserver-1 | ... 21 common frames omitted
axonserver-1 | Caused by: java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: IOException : Tag number over 30 is not supported
axonserver-1 | at jdk.crypto.ec/sun.security.ec.ECKeyFactory.engineGeneratePrivate(ECKeyFactory.java:170) ~[jdk.crypto.ec:na]
axonserver-1 | at java.base/java.security.KeyFactory.generatePrivate(KeyFactory.java:389) ~[na:na]
axonserver-1 | at io.grpc.util.CertificateUtils.getPrivateKey(CertificateUtils.java:88) ~[grpc-util-1.68.2.jar!/:1.68.2]
axonserver-1 | ... 25 common frames omitted
axonserver-1 | Caused by: java.security.InvalidKeyException: IOException : Tag number over 30 is not supported
axonserver-1 | at java.base/sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:135) ~[na:na]
axonserver-1 | at java.base/sun.security.pkcs.PKCS8Key.<init>(PKCS8Key.java:95) ~[na:na]
axonserver-1 | at jdk.crypto.ec/sun.security.ec.ECPrivateKeyImpl.<init>(ECPrivateKeyImpl.java:78) ~[jdk.crypto.ec:na]
axonserver-1 | at jdk.crypto.ec/sun.security.ec.ECKeyFactory.implGeneratePrivate(ECKeyFactory.java:245) ~[jdk.crypto.ec:na]
axonserver-1 | at jdk.crypto.ec/sun.security.ec.ECKeyFactory.engineGeneratePrivate(ECKeyFactory.java:166) ~[jdk.crypto.ec:na]
axonserver-1 | ... 27 common frames omitted
Contexte
- Axon server is started from a docker container
- The error only occures when I enable SSL on GRPC :
axoniq.axonserver.ssl.enabled=true
axoniq.axonserver.ssl.cert-chain-file=/axonserver/tls/fullchain.pem
axoniq.axonserver.ssl.private-key-file=/axonserver/tls/privkey.pem
- the certificate is générated using Letsencrypt
- The setup was working until yesterday I update debian to 12.9 (The last version may be 12.8)
Attempts to undersand
- Check the certificate files : openssl x509 -noout -text -in fullchain,pem : Certificate is valide
- Regenerate Eliptic certificate : same error
- Regenerate Cert with RSA 4096 : same error
My request
- Help me make meaning of the error message
- Help me start the axon server with SSL.