Replay events from a Postgres JdbcEventStore

Imported from Google Groups
1

Hi all.

I’m testing a replayer they will have to replay around 2 million events once we run it in production, until now I’ve been testing replaying one aggregate at a time for development and testing, but now that it seems stable enough I’m trying the JdbcEventStore.visitEvents without criteria, like this:

((EventStoreManagement) expensefloEventStore).visitEvents(domainEvent -> {
    try {
        isolatedEventBus.publish(domainEvent);
    } catch (Exception e) {
        log.error(domainEvent.getIdentifier(), e);
    }
});

The diference to the “aggregate one at a time” is only the crteira:

CriteriaBuilder entry = ((EventStoreManagement) eventStore).newCriteriaBuilder();
Criteria criteria = entry.property("aggregate_identifier").is(uuid);
securityService.runAsAdministrator((id) -> {
    ((EventStoreManagement) expensefloEventStore).visitEvents(criteria, domainEvent -> {
        try {

            isolatedEventBus.publish(domainEvent);
        } catch (Exception e) {
            log.error(domainEvent.getIdentifier(), e);
        }
    });
});

So now what it happens with no criteria is:

  • goes ok until PreparedStatementIterator
    ResultSet resultSet = statement.executeQuery();
    being the statement

select event_identifier, aggregate_identifier, sequence_number, timestamp, payload_type, payload_revision, TEXT(payload), TEXT(metadata) from axon.expenseflo_domain_event_entry e ORDER BY e.timestamp ASC, e.sequence_number ASC, e.aggregate_identifier ASC

  • it spends 20 min. executing it

  • it returns 1 (!!!) event

get’s the next iteration that also takes 20 min. and returns 1 event…

I also tried to play with the batchSize param but to no effect, I think Postgres does not support batchSize?

I’m using 2.4.6, am I doing something wrong or misusing the event store?

Cheers.

2

Would be interesting to see the execution plan of that query, do you have an ordered index with these three columns?

3

I don’t, I’ll try to do that. In the meantime I noticed that someone (most certainly me) had set the batch size to 1 for a reason I don’t know (or don’t remember). I’m playing with that now and I also have changed the EventSqlSchema to add a LIMIT XXXX clause to whereClause in fetchFiltered of the eventEntryStore, but I don’t actually if that is a good idea and/or if t’s going to screw something…

I’m also reading some documentation because I don’t know exactly what that does (at first it looks to me just another way of LIMIT).

I also added a Criteria to only get the aggregate types I’m really interested…

But for a batch size of 1000 it was taking 4 min+ which is still too slow for 2 million events…

I’ll test some more with that index in place.

Thanks a mil for your help’

Cheers.