Monitor event transformation progress

Axon Server
1

Hi
is there a way to monitor the progress of an event transformation that is in state APPLYING?
I started an event transformation (to upcast events persistently) on an AxonServer cluster.

When listing the transformations it is still in APPLYING state and in the AxonServer logs I see logs like the following regularly:

{
  "timestamp": "2026-06-03T09:13:26.990+02",
  "level": "INFO",
  "thread": "boundedElastic-174",
  "logger": "io.axoniq.axonserver.eventstore.transformation.apply.TransformationApplyAction",
  "message": "Applying transformation: 8fc9b264-8d99-49b7-9518-ef091dc427fc",
  "context": "default"
}
{
  "timestamp": "2026-06-03T09:13:26.990+02",
  "level": "INFO",
  "thread": "boundedElastic-174",
  "logger": "io.axoniq.axonserver.eventstore.transformation.apply.DefaultTransformationApplyExecutor",
  "message": "Starting to apply transformation 8fc9b264-8d99-49b7-9518-ef091dc427fc to context default",
  "context": "default"
}
{
  "timestamp": "2026-06-03T09:13:48.040+02",
  "level": "INFO",
  "thread": "boundedElastic-180",
  "logger": "io.axoniq.axonserver.eventstore.transformation.apply.DefaultTransformationApplyExecutor",
  "message": "Transformation 8fc9b264-8d99-49b7-9518-ef091dc427fc applied successfully to local store.",
  "context": "default"
}
{
  "timestamp": "2026-06-03T09:13:48.040+02",
  "level": "INFO",
  "thread": "boundedElastic-180",
  "logger": "io.axoniq.axonserver.eventstore.transformation.apply.TransformationApplyAction",
  "message": "Applied transformation: 8fc9b264-8d99-49b7-9518-ef091dc427fc",
  "context": "default"
}

The transformation is now running since >20 hours. Free disk space looks fine so far and I did not see any errors.

Is there any way to see more about the progress?
Is the APPLY process stuck?
Maybe I am just too impatient?

Thanks
Klaus

2

Hi again,
seems that my AxonServer cluster (v2025.2.7) is messed up because of the event transformation that is still in APPLYING state.

One server node seems to be stuck with a certain replace event and is unable to recover or continue. This node is now out of sync with the other 2 nodes and I see ~200 replication logs in the logs directory. Axon PConsole says replication is behind.

Luckily this is “only” our Test AxonServer cluster, but this is of course serious issue.

In the logs on that node I see:

{
  "timestamp": "2026-06-03T12:43:15.055+02",
  "level": "WARN",
  "thread": "boundedElastic-13",
  "logger": "io.axoniq.axonserver.eventstore.transformation.cluster.logentryconsumer.ReplaceEventLogEntryConsumer",
  "message": "Unsuccessfully accepted replace event [token 293008] action for transformation 96b6c755-31b8-4ae9-9de9-22a7a8907206 in context default.",
  "context": "default",
  "exception": "io.axoniq.axonserver.filestorage.impl.FileStoreException: Cannot reset to value higher than the last index\n\...."
}

The exception is:

io.axoniq.axonserver.filestorage.impl.FileStoreException: Cannot reset to value higher than the last index
	at io.axoniq.axonserver.filestorage.impl.BaseAppendOnlyFileStore.lambda$reset$2(BaseAppendOnlyFileStore.java:55)
	Suppressed: java.lang.Exception: #block terminated with an error
		at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:104)
		at reactor.core.publisher.Mono.block(Mono.java:1779)
		at io.axoniq.axonserver.eventstore.transformation.requestprocessor.LocalContextTransformationStore.storeStagedActions(LocalContextTransformationStore.java:78)
		at io.axoniq.axonserver.eventstore.transformation.requestprocessor.LocalContextTransformationStore.save(LocalContextTransformationStore.java:63)
		at io.axoniq.axonserver.eventstore.transformation.requestprocessor.TransactionalContextTransformer.lambda$perform$13(TransactionalContextTransformer.java:166)
		at io.axoniq.axonserver.eventstore.transformation.requestprocessor.TransactionalContextTransformer.lambda$executeInTransaction$0(TransactionalContextTransformer.java:56)
		at reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:61)
		at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
		at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539)
		at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)
		at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)

These exception now appear every minute and the node seems broken.

Any help is appreciated.
Can we interrupt or cancel the event transformation if it’s already in APPLYING?
Can I “repair” the broken node?

Klaus