tracking processors status

Hi,

From an operational perspective, we want to have insight in the TrackingProcessor status.
We have an endpoint, which reports the status, and we can aggregate this information for all processors in a UI.

This is how the json looks like now [1]:

When AxonServer is in debug mode, it reports how far it is with serving the events. (I my interpretation is correct):

axonserver_1 | 2019-03-05 11:30:25.366 INFO 7 — [event-stream-40] i.a.a.l.EventStreamController : Start tracker from token: 1127686

axonserver_1 | 2019-03-05 11:30:25.405 INFO 7 — [event-stream-40] i.a.a.l.EventStreamController : Done processing backlog at: 1128186

Now I wonder, why the processor reports it is 'caughtUp, while events are still being served?
Could it be this only applies when ‘replaying’ events?

Regards, Christophe

MacBook-Pro:axon-props christophe$ http :8030/axon/processors/

HTTP/1.1 200

Content-Type: application/json;charset=UTF-8

Transfer-Encoding: chunked

[

{

"name": “eu.blox.engine.asset.query”,

"status": [

{

"caughtUp": true,

"replaying": false,

"segmentId": 0,

"trackingToken": “IndexTrackingToken{globalIndex=1127617}”

}

]

},

{

"name": “eu.blox.engine.product.query”,

"status": [

{

"caughtUp": true,

"replaying": false,

"segmentId": 0,

"trackingToken": “IndexTrackingToken{globalIndex=1128032}”

}

]

},

{

"name": “eu.blox.engine.account.query”,

"status": [

{

"caughtUp": true,

"replaying": false,

"segmentId": 0,

"trackingToken": “IndexTrackingToken{globalIndex=1127416}”

}

]

}

]

Hi Christophe,

Caught up returns true as soon as it’s local stream of events does not have any further events to handle.
If this occurs, it’ll wait for one second until it polls again.
What I believe you’re hitting here is that tiny window where the Tracking Event Processor thinks it’s caught up, whilst in essence there are still events coming.
This situation is especially true in a live system of course, as there are no guarantees that it will be caught up forever since new events can be published every second.
It’s thus safer to assume caught up means that it’s at the end of the stream or near the end of the stream.
I recall that that’s initially how this functionality was propagated too.

So, in short, the caughtUp flag is not just for when you’re replaying, it’s to signal you’re near/at the end of the event stream.

Hope this helps!

Cheers,
Steven

Thanks for the explanation Steven, I do however notice after some further testing, caughtUp is also true when replaying and starting from the tail of the event store.
It’s basically always true, I will report an issue on this.

Hi Christophe,

That doesn’t sound like intended; if you’d be willing to create an issue for that for us, that would be very helpful!
Which version of Axon are you on again, 3.4.x, right?

Cheers,
Steven

This is on Axon 4

Hi Christophe,

i think I encountered the same behaviour in Axon 4 as you did. Did you file an issue as suggested by Steven? If so, It would be great if you could share the link so I can have a look if it has been resolved already (did search on github but didn’t find any related issue)? If not, did you do further research or get insights you could share?

Best Regards,
Jakob

Hi Jakob,

I didn’t file an issue yet, as I wanted to test this a bit more, but this is now prevented by other work…
So, sorry no more info regarding this.

Rgds, Christophe

Not a problem, thanks for responding. I’ll post to this thread if I find out more on this.

Best Regards,
Jakob

FYI I just filed an GitHub issue that looks like the same behaviour as I see in our environment: https://github.com/AxonFramework/AxonFramework/issues/1505