Data import with commands

Hi there,

this question is slightly off topic as it is more about concepts, but I’m a (happy) Axon user, so I ask here for insights…

I’m facing the problem of data import from 3rd party applications using commands.

Imagine an organization aggregate consisting of an id, organization name, organization type, a address and so on. The import data comes as snapshots of data of varying granularity, e.g. a line in a CSV file giving the name and type of a to be imported organization. Another one might give just the id and a address. These formats are not fixed, so a flexible approach is needed.

Right now I have one create, edit, and delete command for each aggregate used in the import process. This works quite good if all data is at hand in the importer. However, as mentioned above the CSV file might just not transport the organization address in the current import data source. The e.g. edit command expects a complete snap shot of the organization containing all data, however. What to do in this situation without typing Java code like crazy?

  1. I might write a command for a concrete input format, e.g. a edit command taking only the name and the type. However, this leads to many commands for different scenarios and a lot of lines in the command handler. Not nice.

  2. I might add a boolean flag for each field in the command marking it as to be processed or not. Also not so nice, as this boils over to events and ultimately leads to a complex command handling and an even more complex service for building the read side.

  3. I might load data from the read side to complete the command. This might have concurrency issues as I don’t know if e.g. the address is the right one or a new one is currently processed.

Have you dealt with such problems in the past? Did you find another viable solution which I have not thought of? Or have you got any tricks to make a solution listed above appear more viable?

Thanks!

-Sebastian

Hi Sebastian,

I am just assuming that the company information you’re gathering is the type of information that is only changed occasionally? Where it revolves around data rather than behavior.

In that case, I can image you don’t want to spend to much time creating NameChangedCommand/NameChangedEvent, AddressChangedEvent/Command, etc. what about creating a PartialCompanyUpdate command and leaving the unknown fields null? You could have a similar event. The difference with the ‘normal’ events is that nulls don’t mean data should be removed. Alternatively, but that’s a matter of taste, you could just use the existing commands and events and use Null to indicate information is unknown/unchanged and an empty String to indicate data should be removed.

Just hope this gives you some ideas.
Cheers,

Allard

Hi Allard,

thanks for your input. You are indeed right that the import just deals with data… it is not really behaviour driven.

Your two solutions are an interesting alternative. I think experience pays off big time here :wink: Thanks!

Keep you posted how it turns out!

Cheers,
Sebastian