'Command Side Projections' for uniqueness constraints

Axon Framework
,
#1

There is a common situation that at the point of the creation of an aggregate we need to do a uniqueness check.

For example for the creation of a User aggregate we want email fields to be unique across all aggregates (example taken from [1]).

I believe that this problem is a general instance of the larger ‘Set Validation’ problem mentioned in [2]. To resolve the problem we would need “to validate the uniqueness against an immediately consistent datastore” (e.g. database table with unique constraint on the email column) before dispatching the command [2].

There are many ways to implement this within Axon:

  • In [3], Steven Grimm describes “the old-fashioned way” of placing a a constraint on a SQL
    table in the query model.
  • Working with a distributed lock would also work. I think [4] is a good video explaining how to
    do this in Spring.
  • In [5] @allardbz lists techniques you could use including using a 2 step process with a Saga.
  • In [6] @Sebastian_Ganslandt1 suggests using “an interceptor that checks the incoming create-commands against this query-model and rejects the command if it contains a duplicate”.
  • @Ivan_Dugalic in his answer in [1] wrote that you can also use ‘command side projections’.

If I was trying to solve this using a Saga then could I create a Saga which has @StartSaga on a @CommandHandler rather than an @EventHandler? The saga would then send further commands until finally the last @CommandHandler (annotated with @EndSaga) either creates the User or does not. Using a saga with events does not feel right for this purpose (but using commands seems to make more semantic sense to me).

Or if trying to solve this using ‘command side projections’ how exactly does that work?

  • Do we essentially just create a @Component which has a @CommandHandler that as part of
    its logic takes some sort of lock and validates the constraint?
  • How exactly would you implement this?

Are there any code examples illustrating this ‘set validation’ pattern using command side
projections in Axon that I could use to understand?

Regards,
vab2048

[1] Route one command to multiple aggregates
[2] Validating Uniqueness
[3] Validating Uniqueness
[4] https://www.youtube.com/watch?v=firwCHbC7-c
[5] How can I prevent Asynchronous EventHandler handles events that should be rollbacked
[6] Best appoach to pre creation bus rules for aggregate

1 Like
#2

Hi vab2048,

We get this question often and I am working on a blog about this topic which will be published soon. This blog elaborates more on the answer that @Ivan_Dugalic gave. To give you a sneak preview this is a link to code samples that I use for this blog Set based consistency validation with Axon.
I’ll let you know when the blog has been published.

Kind regards,

Yvonne

5 Likes
#3

Hey vab2048,

In addition to her blog, Yvonne will be talking about set based validation in a podcast with me that will be released soon. We’ll keep you posted. :slight_smile:

3 Likes
#4

Hello vab2048,

The blog has been published and you can find it here

2 Likes
#5

Hi,

How will the look-up table solution scale, if we’re using multiple instances of the commands processors?

Thanks,
Eliezer

#6

Hi Eliezer,

You can only have one command handler for a command. Are you in a situation where you need this lookup table for a lot of commands or do you have an example?
Because all rows of the table need to be available (for complete validation) you 'll need to keep the number of columns in this table as low as possible and add indices on the search columns.

Yvonne

#7

Hi Yvonne,
I presume that in a scalable solution, multiple instances of the same command handler can be deployed.
In this scenario, should we use a dedicated lookup table for each instance, or should we use a central one?
Thanks,
Eliezer

#8

Hi Eliezer,

This should be a central lookup table to be able to validate the whole set.

Yvonne

#9

Hi Yvonne,

I’m wondering if this approach also takes concurrency into the account? For example, if 2 commands intending to create the accounts with the same email address arrive at the same time, there’s a chance that both of these commands will pass through the Dispatcher Interceptor validator and when they reach the command handler, 2 AccountCreatedEvents will be raised. One of them will obviously fail to update the lookup table due to a unique constraint violation, but still, I have 2 questions:

  • will this failure cause to “roll back” the even from the EventStore? In other words, is this Subscribing Event Processor executed in the same UnitOfWork that the command that originally raised the event?
  • even if the lookup table and even store updates participate in the same UnitOfWork, they are still done against different storages. Isn’t this sort of a distributed transactions problem then (which again, can not guarantee the immediate consistency)?

Kind regards,
Beka

Same surrogate ID for aggregates in different bounded contexts
Same surrogate ID for aggregates in different bounded contexts
#10

Very reasonable scenario Beka. In essence, this could still occur.
However, the chances are greatly minimized with what @Yvonne_Ceelie has shared in her blog post.

Now, towards your exact questions.

  1. When using the SubscribingEventProcessor the same thread used to invoke the command handler is used to invoke the event processor’s event handlers. Internally, this means the same UnitOfWork is indeed used. Thus if storing the field in the set (for validation) fails exceptionally, that would roll back the entire UnitOfWork. Doing so thus also roles back storing the event in the event store.
  2. This depends on how you have set up the storage solution for your events and the set, to be honest. You could in essence regard the event store as the consistency solution for your Command Model (if you are event sourcing, that is). From that perspective, it wouldn’t be overly wrong to include the consistency-set to the same storage solution. However, as Axon Server users this wouldn’t be possible at all, since Axon Server is only an Event Store. Hence you would be left with two distinct storage solutions. If the consistency-set is actually stored in a database is a different choice of course. Maybe in-memory would be just fine, albeit that this approach requires a warm-up whenever you shut down your system.

You could adjust the process even further (for which we’re thinking about a blog-update actually). Think of having an interceptor which “locks” an entry for being used by other threads on the set. Then a TrackingEventProcessor would come after your command process (in thus a different UnitOfWork) which “sets in stone” that the field/id/email is really used. There are likely still some caveats in this approach I haven’t thought off (as I am still trying to figure out a fool proof approach).

At any note, I think it is good to deduce what level of consistency you really need in your application. going the full 100% automation (for anything really) is in most scenarios extremely tasking with any approach. Resolving about 80 to 90% could just be enough, with a manual approach (thus dedicated commands/events to revert the wrongfully used ID) next to it to resolve the problem when/if it occurs.

1 Like
#11

You could adjust the process even further (for which we’re thinking about a blog-update actually). Think of having an interceptor which “locks” an entry for being used by other threads on the set. Then a TrackingEventProcessor would come after your command process (in thus a different UnitOfWork ) which “sets in stone” that the field/id/email is really used. There are likely still some caveats in this approach I haven’t thought off (as I am still trying to figure out a fool proof approach).

I can think of a scenario when the lock may never get released, for example when command processing fails after the dispatcher interceptor locks the entry, but before the event is persisted to the store (“set in stone”). In such a case, further attempts of command re-delivery (dispatching) won’t be possible until this lock is released. One could introduce a lock expiration mechanism, but still not an ideal solution.

I’m thinking if it’s actually worth doing the extra work if there is no 100% bullet-proof solution. Perhaps the approach where compensating events are applied should be enough, which has to be coded anyways, to cover that very extreme cases.

You could in essence regard the event store as the consistency solution for your Command Model (if you are event sourcing, that is). From that perspective, it wouldn’t be overly wrong to include the consistency-set to the same storage solution

On the other hand, I assume that event store(s) already have a concept of a unique index. Maybe it would be practical to expose an API for framework users? (does seem a bit messy though)

#12

From a very high level perspective, I would like to see a solution in which I can add an annotation to a property of an event, and the framework will make sure it is unique.
From the low level perspective, maybe the solution should imitate the pattern of a DB index, along the lines of the ideas discussed in the blog post, and this thread.

#13

You can go a long way with the shared solution in the blog and what I’ve stated, but doing everything is indeed rather tricky. If you anticipate this to happen a lot (concurrent usages of a field that should have a uniqueness constraint), then adding this would make sense. If you also anticipate this to not be sufficient, having the compensating action solution is indeed the way to go.

Added, I like to thank you @Beka_Tsotsoria and @eliezerb for your insights in the matter. We are always on the lookout to improve Axon and we simply cannot do that without user insights. At this point, I still find it hard to state whether we will have a solution inside the framework to deal with this. I will, by personal conviction, never rule this out.