I’d like to make sure I’m understanding event segments correctly.
Based on the documentation here, my understanding is that event segments are a mechanism whereby each TEP (TrackingEventProcessor) instance distributes events to the threads controlled by that instance. This ensures that two different threads on the same TEP instance don’t both try to process the same event.
Have I understood what event segments do correctly?
If I have, great. However, while this says what it does, the documentation does not actually explain what an event segment is. What exactly is an event segment?
Hi @fjx, you’ve hit a piece of the Reference Guide that could have used some love when we updated the Event Processors page. Sadly, we (read: I) missed updating this page as such not providing the valuable reference that could help you to better understand what this is.
Now, let me go to the questions at hand to clarify things:
You’re right. The segments ensure that two or more threads do not cover the same section (segment) of events from the event stream.
A Segment refers to a portion of the Event Stream. What this portion is, is deduced by the SequencingPolicy. Furthermore, such a Segment is stored in a TrackingToken, for which you can find the documentation here.
They thus provide a means to break up the Event Stream into several sections, allowing for parallel event processing. The Reference Guide has this to say on the subject.
I hope this clarifies things for you, @fjx.
If not, feel free to keep commenting under this thread!