Initial import from rdbms

Imported from Google Groups
#1

Another evaluate/newbie question.

I have a postgres database with all my data for the last 10 years and want to move to CQRS. My application is dependent on time already so everything in the database has creation dates. When/if moving to Axon I need these dates. One simple solution is obviously to post the dates with the events much like in the rdbms but from here on I would rather use the events timestamp as it fits better with Axon and CQRS.

I have looked in the Axon code (4.0) and see that a message is tagged with the clock.instant() in the GenericDomainEventMessage. I could not see any obvious way other than possibly writing my own serializer to ‘post-date’ the events, would that break anything else if I did that?

What would you suggest for migrating a current rdbms to ES?

Cheers,
Fredrik

#2

Hi Fredrik,

my own very first Axon project started exactly like this. Your typical 3-rd normal relational database application that needs to migrate to an event sourced structure. Most of the entities carried a date that allowed to retroactively apply some events at that date.
That’s also where, in hindsight, some things went wrong. We applied those events with a retrofitted date, but not all events. So if we were to do a time-based replay, events for the same aggregate would be applied completely out of order.

So only retrofit timestamps of events if you can do that for all events. Axon takes the current time from the GenericEventMessage.clock, which is a non-final public field. Guess why ;-). It defaults to the system clock and the UTC timezone. Do be careful when changing it, as you might run into concurrency issues.

Alternatively, you can consider the timestamp the time of that “fact” that the data was inserted in the new system. In this case, your event payload would also contain a timestamp, which is the intended time of the change. For example, in a system that registers people’s address, you could send an address change at time X, while the actual address change is done at time Y. In this case, X is the timestamp of the event (the time at which you were made aware) and Y is in the event (representing the time at which the change will take effect).

Whether one or the other scenario fits your situation best, depends on the domain you’re in. The second approach is nicer, but only if it’s also considered an advantage after the import. If it’s only for the import, backdating the events could be a cleaner approach.

Hope this helps.
Cheers,

Allard

#3

Thx for a thorough answer Allard,
you confirmed my ideas and gave me more to consider, you are absolutely correct about getting the ordering of the events correctly.

I listened to a talk by Greg Young where he raised the idea of multi timed events where events could be considered to have multiple dates for trying out different scenarios in projections. In my implementation, if adding my own dates I will get almost this solution as I will be able to see what happened from the domain perspective by using my own dates, and when they really happend by using the events default time thus giving me an opportunity to actually see the time of the migration for instance even thought that might not be exactly reflected in the domain dates. Exactly as you also explained I can add more dates for what will happen in the future. Exciting times :slight_smile:

Thanks for your great answer.

Cheers,
Fredrik