BIGSERIAL and commit order of globalIndex

I’ve been browsing the source code of Axon Framework’s Event Store persistence layer, to find out whether a common but little-known concurrency issue applies here that possibly results in missed events for consumers.
Namely, most implementations of Event Stores that I’ve encountered rely on a global serial number for events, and a “token” saved somewhere that tracks the consumer’s progress relative to that serial number.

This assumes that events become visible in the serial order, that is, the following invariant holds:

Given Domain Events a and b

If a.globalIndex < b.globalIndex
Then a was visible earlier or at the same time as b

This basically means that the Event Store prevents this sequence of occurrences:

  1. a.globalIndex is allocated a value (say, 1)
  2. b.globalIndex is allocated a greater value, as it should (say, 2); whether it has gaps relative to a’s index or not (as would happen with PostgreSQL’s sequence cache pre-allocating numbers) is unimportant
  3. b is committed and made visible to SELECTs
  4. a is committed and made visible to SELECTs
    For purposes of discussion, I call it the “sequential visibility principle”, and the situation above would violate it it.
    This principle is not ordinarily true in SQL databases: it is possible for transactions to commit in any order, which is totally independent of order of sequence number allocation. This is because sequences are unaffected by transactions.
    It is trivial to cause this by hand using 2 separate connections to a database (connections are marked as a and b, assume seqnum is a BIGSERIAL):
  • a: BEGIN;
  • b: BEGIN;
  • a: INSERT INTO seqnums (seq) VALUES (DEFAULT);
  • b: INSERT INTO seqnums (seq) VALUES (DEFAULT);
  • b: COMMIT; – note: we run this on connection “b” first, violating the invariant as described
  • a: COMMIT;

Causing a similar scenario, but in an Event Store, has an undesired consequence: when token-based consumers “tail” the Event Store, they may skip the earlier sequence number if it becomes “visible” to other transactions (in the MVCC sense) later.
Consider Domain Events with these sequence numbers present at some moment, t0:
[ 1, 2, 3 ]

Now, we append two events from totally separate entities. However, assume they COMMIT in the reverse order (which is totally fine by the database).
There exists a moment in time, t1, where the Event Store looks like this:
[ 1, 2, 3, 5 ]

Obviously, after some time, the transaction for sequence number 4 will end up committing and making the entries visible for SELECTs, or rolling back, in which case it is not a concern.
However, if at moment t1 a token-based consumer were to cycle through the Event Store, upon reaching number 5 it must think it has read the entire thing. Therefore, it will save its last-seen token (5), and will never go back to 4, at least under these assumptions.

My question is this: since Axon with the JPA-based Event Store relies on the globalIndex (BIGSERIAL) column for its tokens, what exactly prevents the scenario outlined above from happening? Do we employ locking (table locks or advisory locks) anywhere?


ye,s we definitely take into account that insert order and commit order aren’t the same on all database types. In Axon, the event store will read from the last known sequence upwards on each batch, but it keeps track of the “gaps” that it has encountered. These gaps may indicate yet-to-be-committed entries, but could also mean rolled back entries or simply sequences that have an increment of more than 1. There are mechanisms that clear those gaps when there are too many or if they have become “too old”.

That said, it’s always more efficient to use an Event Store implementation such as AxonServer that doesn’t have these problems. Checking gaps will always come with a little bit of overhead.