createHeadToken always has an empty gap list

Imported from Google Groups
#1

Hi,

I noticed that in the createToken method of JpaEventStorageEngine the gap list is always empty. Is this an oversight? (The same can be said about other implementions such as JdbcEventStorageEngine by the way.)

private TrackingToken createToken(List<Long> results) {
    if (results.size() == 0 || results.get(0) == null) {
        return null;
    }
    return GapAwareTrackingToken.newInstance(results.get(0), Collections.emptySet());
}

This method is used in other methods such as createTailToken or createHeadToken. I am running into issues with this when comparing the head token to the token of a processor in order to see if the processor is running behind: as soon as the processor has any gaps the covers method of GapAwareTrackingToken returns false.

Regards,
Jens

#2

Hi Jens,

no actually, it is not an oversight. This method is provided to allow you to start tracking from the head of a stream.
How do you compare the tokens? It is probably enough to only use the getIndex() value from the token.

Hope this helps.
Cheers,

Allard

#3

Hi Allard,

Your suggestion of using getIndex() would solve my false positives indeed. However, using “processorToken.covers(headToken)” seemed safer to me since I was expecting that this would also warn me if the processor had a gap that it shouldn’t have compared to the event stream. What are your thoughts on that? Am I being too paranoid?

Regards,
Jens

#4

Hi Jens,

sorry for the late reply. I got swamped with messages in the new year.
It’s not up to me to call anyone paranoid. It’s all about what you want to measure.

If it’s about the relative distance of a processor compared to others or the head of the stream, I would say the index is enough. A token with gaps will never “cover” the head token, unless gaps are “cleaned” prior to the check. Any remaining gaps will still cause the “covers” to return false.

Cheers,

Allard