@EventHandler Method is not invoked

First of all, many thanks for the great framework, I am new to Axon Framework and I feel I am already falling in love with it. In short, here is my setting and problem:

Setting

  • Axon Server SE.

  • Spring Cloud App as gateway, distributing incoming REST requests to mircroservices (Spring Boot Apps) via Axon Framework.

  • One of the microservices (so far the only one implemented) uses MongoDB for storing/retrieving documents. I have set up a MongoDB based token store and use the same MongoDB as for documents (as suggested in Axon Guide).

    • The ‘Document’ aggregate has proper 'CommandHandler’s and 'EventSourcingHandler’s for creating and updating documents.

    • A ‘DocumentProjection’ service – a Spring Boot Service with ProcessingGroup(“documents”) – has several 'EventHandler’s which are applied events by the Document aggregate’s command handlers, and care about maintaining the projection database, MongoDB it is, and retrieving documents via fetch events, accordingly.

The described setting above is nothing special, it follows the Axon examples such as giftcard, hotel reservation, etc. Now, here is my problem.

Problem

  • During the development I have refined the ‘Document’ aggregate and restructured things such as event classes. And somehow I came several times into the situation, that my event handlers in DocumentProjection service did not get invoked anymore. I have deeply dived into axon sources and could not find the reason. But I could make an observation: every time I ran into this problem, I deleted the trackingtokens collection in MongoDB and restarted the service, and suddenly all events began to be delivered as expected, also those, which I had missed when I created or updated documents which did not arrive before.

My Question

Is there something special I have to consider to avoid such a situation? It seems that the trackingtokens get out of sync with the axon server, and I could not understand, how I managed to break the trackingtokens. Could that have to do with playing with event classes?

Hi @boto, welcome to the forum!

Let me focus on one specific sentence in your question, as that stroke me as interesting:

This sentence to me seems like you were expecting Axon Framework to automatically rehandle events after you have adjusted their format.
Is that a correct assumption from my end?

If so, I can be short about it: Axon Framework does not automatically replay events once you have changed the structure.
If you do require a conscious replay of events, you should use Axon’s Replay API.

This would mean you’d first retrieve the correct StreamingEventProcessor (in your case, the processor with the name documents, as follows from the @ProcessingGroup("{...}") annotation) and invoke the StreamingEventProcessor#resetTokens operation.

This will let your processor start from the beginning of the event stream to replay your events.


Granted, the above only makes sense if my earlier assumption is correct.
So, feel free to add another reply stating whether this is the case.

Hi @Steven_van_Beelen, thanks for your reply.

This sentence to me seems like you were expecting Axon Framework to automatically rehandle events after you have adjusted their format.
Is that a correct assumption from my end?

Nope, what I meant was that new events issued as side effect in new incoming commands (using the AggregateLifecycle’s apply method) did not arrive in the corresponding event handlers. I was not interested in past event, but only in new ones.

Cheers
Boto

Did you, as part of adjusting the Document aggregate and the events, adjust the actually storage of your events too?
So, did you do things like:

  • Clear out the event store?
  • Point your application to a new event store?
  • Introduce upcasters for your events?
  • Clear out the event store?

I cleared also the event store (because I started the axon server in debug mode), and saw no change.

  • Point your application to a new event store?

No, the event store config was not changed.

  • Introduce upcasters for your events?

Nope. No event casters were used at all.

One of the things I have changed was changing the event class itself. I don’t know if that was the reason for the problem.

I think, I have definitely messed up something and that explains the observation. I wished to get some logs from axon framework to know, what exactly I have messed. I have set the log level of axon framework to trace and still see no anomalies. Are there any other ways to increase the log details?
As far I am on the early development phase, it is not a big deal to erase the trackingtokens collection every time I see missing event arrivals. However, if I run into a similar problem later when the system is on production environment, then erasing the trackingtokens collection would be something I would like to avoid.

Alright, then I got the predicament.

When you clear out the Event Store but do not change the stored tokens, your tokens point to a position your clean Event Store hasn’t reached yet.

See it as such:

Your current event store contains 100 events.
If your Event Processors are on track, they’ve handled all 100 events.
This makes their tracking token point to position 100.

Now you stop your application, remove all events from the store and start it up again.
When new events are stored, they’ll start at position 0, incrementing from there.
However, for the Event Processors, it seems like they’re already on track.
They are actually 100 events in front of the work!

Hence, they won’t show you the “new” events simply because the tokens dictate the processor has already processed those events.

This means that whenever you’re in a development phase, and you’re clearing out your Event Store, it’s recommended to clear out your query models and tokens too.

Ah got it, it makes sense. Thank you for the hint :slightly_smiling_face:

1 Like