Domain evolution and event sourcing

Imported from Google Groups
#1

What is the preferred approach to evolving the domain given a system
that has gone live and the domain event for the various aggregates
have been stored in the event sourcing repository. I will try and
illustrate this with as example.

I start with a domain that has two entities a Account and an Account
Type. The Accout Type may be Credit Card, Debit, Cheque etc. After the
system has gone live a large number of accounts have be created of the
various types. So the event store would look like

AccountTypeAddedEvent (Credit Card)
AccountTypeAdded (Debit)
AccountTypeAdded (Cheque)
AccountAddedEvent (type = Credit Card)
AccountAddedEvent (type = Debit)
AccountAddedEvent (type = Debit)
AccountAddedEvent (type = Debit)
AccountAddedEvent (type = Cheque)

After speaking to the business analysts and users we determine that
the domain needs to change from:
Account has an Account Type
to
CreditAccount, DebitAccount and ChequeAccount extend from Account

I am guessing that it is a matter of replaying the domain events from
the repository by associating then with the new aggreates or are there
other alternatives? I think that the domain could evolve substantially
after the system has gone live and the evolution could be quite
complex. Does axon support these situtation or I going down the wrong
here?

#2

Hi,

this example of migration of events is one that you’re very likely to encounter.

Axon 0.6 doesn’t provide support for this yet, but the 0.7 version does. To support changes in your event stream, you can configure “upcasters” in the event serializer. These upcasters work with an intermediate serialization format. The XStream serializer uses dom4j documents as intermediate, mostly due to the nice API and good performance compared to other xml processing libraries.

For you example, you would rewrite an event of format
Credit…more data…
to something like
…more data…

The 0.7 also adds a “revision” attribute to the root element. The upcasters can use this revision to decide which action they need to take. Each time an event changes format, you should increase this revision number.

Hope this helps.

Cheers,

Allard

#3

Hi Allard,

Currently working on versioning for the Axon Auction Example I think
just to version the events is not enough.

In fact we need a good versioning scheme for all 3 layers of a typical
J2EE application:

1) A client sends a command to the Command Server
The client may use an old client JAR and therefore sends old commands.

2) The event changed and there are old versions stored in the
repository (Command Server)
This is about the mentioned "upcasters".

3) Messages resulting from events are sent to message clients (JMS)
There may be message clients that aren't up-to-date.

I know 1) and 3) are not part of the Axon Core but all layers have
some kind of serialization and face version change.

The best solution I found so far is to use a version in the class name
like "RegisterUserCommandV1". Old commands could be moved in an "old"
package but this requires some kind of name translation within the
deserialization process.

Example:

1) We receive a
"org.fuin.auction.command.api.base.RegisterUserCommandV1" command

2) Checking some kind of version table we discover it's an old command

3) The command is then deserialized into a
"org.fuin.auction.command.api.base.old.RegisterUserCommandV1" class
(Note that "old" package that is different from the package sent by
the client!)

4) A converter converts the ".base.old.RegisterUserCommandV1" into a
".base.RegisterUserCommandV2" command.

5) The command handler executes the "RegisterUserCommandV2"

- Step (2) requires a "Command Version Registry" concept
- I think step (3) can be done with XStream's alias feature, not sure
about other protocols (SOAP, Hessian, Google protobuf...)
- Step (4) needs some kind of (generic) Converter<SRC, DEST>...

Cheers,
Michael

#4

Hi Michael,

Api versioning is complex, and most of all very painful. Unless there is an extremely strong business case for it, I would not allow any type of API versioning to happen. With that, I mean that clients should not be accepted unless the “speak the latest version”. In most applications (at least with web applications), the party providing the clients is the same one providing the server components.

If you do need API versioning, for whatever reason, I would be careful not to “leak” this problem into your own domain. In other words, I would never opt for “CommandV1” inside the domain model. My domain knowns Command, no more, no less. If other apps don’t understand that, it’s because they speak another language (or their language doesn’t change as often). This is where typical DDD building blocks come into play: use an Anti Corruption layer. In other words: if you need to version, do it outside of your domain. It’s an interface thing, really.

So my approach would be to just send your commands and events (latest version) to the JMS queue. If applications aren’t able to cope with design changes in the applications, they should use an intermediate layer (i.e. anti corruption layer) to do the transformations. If the business case is strong enough, you might even provide that layer yourself. Especially in a sample application, I wouldn’t go that way. It’s just making it too complex.

Cheers,

Allard

#5

Hi Allard,

Api versioning is complex, and most of all very painful. Unless there is an
extremely strong business case for it, I would not allow any type of API
versioning to happen.

I think this point of view is a major mistake made many developers &
projects. Change is the normal case and not a special one. You may
create a new project within year but it'll be running for 10 or more
years. So what do you think is the more important part of the software
life cycle? I've personally seen many cases where the version of the
web service used was years (!) old. Being backward compatible is a
crucial part of every service layer. Planning the change should be an
integral part of every project and not a "it's painful, don't think
about it" approach.

use an Anti Corruption layer. In other words: if you need to version,
do it outside of your domain. It's an interface thing, really.

Sounds good, but how to do it? For now I'll create a new lower
prioritized use case for the "versioning" theme.

Especially in a sample application, I wouldn't go that way.
It's just making it too complex.

I see the example application more as a blueprint that should contain
all aspects of creating a real world J2EE Axon CQRS application. This
includes versioning for all layers for the above reasons.

Cheers,
Michael

#6

I think I didn’t get my point across properly.

I am not saying that change is bad. Nor am I saying that backward compatibility is an issue that should only be addressed when the problem pops up.

What I am saying that you should have a strong business case if you want to support multiple versions of an API at the same time. If possible, force all your clients to migrate. In web applications, that usually not a big issue, since the UI is provided by the same team as the server components. In that case, you would only need to provide the latest API.

If there is a business case for versioning (i.e. you need to maintain older versions of the API), and I have been in those projects, then you should try to solve that problem outside of the domain model. A domain model is created to solve specific problems. Versioning of the API is typically not one of them.

By abstracting away the internal shape of commands and events, for example using XML, Json or whatever, gives you the ability to change the internal representation (to help you better solve the problems the domain is facing) without changing the external representation. It also allows you to have more external representations than just one, hence support versioning.

In CQRS terms, that would mean that you could have translators to translate “legacy” commands into the form they currently have. If you use SOAP web services, you’d have an endpoint that receives V1 command in XML shape. That would be translated to a Command object and sent on the bus. There could also be a web service (maybe not even soap) that accepts another format, V2, which is translated to the exact same Command object. Then, the domain model will only need to deal with a single command: the last one.

I am a strong believer that the way you communicate with external parties should be carefully extracted away from the way you solve your domain concerns internally. So I am not saying that you should not solve the problem, just that the domain model is not the place to address it in.

Cheers,

Allard

#7

Hi again,

Good to hear you see that point. I agree with you that it's not a good
idea to put versioning in the domain model. I'll find a better
solution for that problem and I'm going to include it in the
example :slight_smile:

The Axon Auction Example is for me a "proof-of-concept" - Is it
possible to build real-life J2EE application with CQRS (Axon)? One of
the problems that may arise is versioning (and in my experience a very
common problem). The blueprint should include a possible solution for
it to show people interested in Axon a way to get things done.

Cheers,
Michael

#8

Hi Allard,

I finally created a small library that handles versioning of
serialized classes out-of-the-box.

Here is an example how it works:
http://www.fuin.org/serialver4j/examples.html

It currently supports:
- Plain Java Serialization
- Hessian
- Burlap
- XStream

Planned:
- Kryo (http://code.google.com/p/kryo/)

(A good benchmark for JVM Serializers can be found here:
https://github.com/eishay/jvm-serializers/wiki/ )

The good thing - It can be used for both tasks:
Versioning Axon Events and versioning Commands (The previous
discussion).

I'll add a "VersioningEventSerializer" that implements Axon's
"EventSerializer" shortly to the Example project. This way Axon now
supports all of the above formats. (If you want we can include it into
the Axon Core)

Cheers,
Michael

#9

Hi Michael,

looks nice! It’s probably a good alternative to the dom4j solution currently provided in the core.

So instead of the XStreamEventSerializer, there should be an implementation for serialver4j (“SerialVer4JEventSerializer” ?) which uses a serial4j class to do the actual serialization.

I like the idea of keeping old class files, but moving them to a “deprecated” jar. It’s a lot friendlier than doing dom manipulation. But then again, having the choice is even better.

Good job! Looking forward to the implementation in the sample.

Cheers,

Allard

#10

I like the idea of keeping old class files, but moving
them to a "deprecated" jar.

Good idea - I added this as a tip to the website.

I also found a way to get rid of the XML file using annotations:
http://www.fuin.org/serialver4j/tips.html

This way it's just a breeze to annotate the old classes with
"@Archived" and "@Deprecated" and move them into a separate JAR.

Looking forward to the implementation in the sample.

Hope to get it done over the weekend :wink:

#11

Hi all,

I added a new "Axon Support" module to the Axon Auction Example.

It will contain all code that is not directly dependent to the Auction
Exmple itself and may be used in other projects.

Currently it contains the VersioningEventSerializer - A new Axon
EventSerializer implementation that supports Plain Java Serialization,
Hessian, Burlap and XStream:

http://code.google.com/p/axon-auction-example/source/browse/trunk/axon-support/src/main/java/org/fuin/axon/support/base/VersioningEventSerializer.java

Cheers,
Michael