Hi Allard
Please read in-line:
the concept of a durable unit of work (one that can carry
on dispatching when a system recovers from a crash) ...
To be clear, in the model I'm describing, there is
no "_A_ durable Unit of Work". Instead, the result
of a successful unit-of-work transaction (i.e. the
first local transaction) is the information about one or
more _Commit_ object(s) persisted durably in the
transaction-log. At the end of the first local transaction,
the main transaction is formally considered successfully
committed; no rollback is needed/permitted
passed this point. It is committed but incomplete.
It may remain incomplete indefinitely until all the
local listeners acknowledge the receipt of all the events
unanimously within the same second local transaction
or someone do something about it manually.
I'll come-back to this later but in this model each
and every event-listeners are local/in-process.
Every remote/out-of-process event-listener must
be encapsulated under a plug-in and register as
a local event-listener.
If the system crashes when some of the listeners
have received the events and some others haven't
received the events, the Commit(s) remain
incomplete. The next round all of the listeners
must acknowledge the receipt, even the ones that
had received the same set of events before. Hence
the need for listeners to be idempotent.
If after a crash and before the restart of Axon
someone changes the configuration of the application
and define a modified set of listeners, the new
set of listeners will be the correct one. In
other words a Commit/transaction-log doesn't
contain any information about the recipients.
In this model, the domain-event gets demoted and
is not the main focus of the event-store. Instead
Commits become the focused permanent citizens
of the event-store, each owning a set of domain
events. And yet Commits are hidden from the
event-store's clients (unless the clients express
interest in them), From the clients vantage-point
it is event-store and not commit-store.
The reason I keep using the term _Commit_ object
is because It links this discussion to the
original documentation explaining the Commit concept.
By definition, a Commit contains a set of domain-events that
happened to a given aggregate during a transactional
unit-of-work as well as any meta-data Axon internally
may need to carry it's operations.
One shortcoming of the approach you outlined below
is the fact that only domain events are stored
in the events store.
The transaction-log (and not the event-store) can
contain anything and everything Axon see fit, even
transient information.
A good Durable UoW should be able to recover everything.
Yes. The transaction-log can contain everything needed
to be recovered on a per-Commit level (not a UoW-level).
Which items may be needed to be saved at the UoW-level?
(other than the callback, which is discussed below)
The only thing that is really unrecoverable
is the callback. Clients should not expect a result
of the command anymore through the Callback.
All the listeners are local/in-process listeners. If the
system or Axon process crashes, the next time around, as
part of the system-startup, the callbacks gets registered
again for the current set of local listeners (which may
or may not be the same set before the crash). If there
are remote/external listeners, it it the responsibility
of their in-process local agent/proxy (registered as a
local listener with Axon) to handle the events.
In your email, you mention that you need local transactions
only, for this process.
Yes. That is one of the main goals of this model.
That's not entirely true. Your second transaction states:
-- The second local transaction pushes the Commit to the
registered event-listeners and (transactionally) marks
the Commit record on the transaction-log as _distributed_.
There is no way to guarantee that the event listeners
and the marking are transactional.
I think I answer this in the following paragraph. If not
please explain.
If the system crashes right after publishing the events,
there is no way to roll that back.
When the first local transaction is succesfuly committed
(and therefore its entry in the transaction-log is persisted)
the main transaction is considered successfully committed,
even before attempting to call-back any listener or writing it
to the event-store. Off-course it is not completed yet, but it
will eventually become completed. There is no way and no
need to rollback a committed transaction. If the system
crashes in the middle of publishing the event to the local
listeners, i.e. before the successful commit of the second
transaction (which means the transaction-log entry exists
but not marked as _distributed_), after the system resurection,
pushing the events starts all over again. Eventually, all the
listeners will acknowledge together, in the one and
the same local (second) transaction. At that time
pushing the events is considered to be completed, but
still the main transaction is not completed yet -- until
and unless the third transaction successfully commits.
At that time the Commit objects are safe and sound living
in the event-store and the transaction-log is deleted.
That's only possible if the transaction log is located
in the same distribution mechanism as the event listeners.
So still, you need a transaction that spans over both
these systems.
In this model there is no such a thing as
remote/external/foreign etc. Everything and anything
is local. If some remote/foreign/out-of-process systems
are involved (such as a message-oriented-middleware like
ActiveMQ or RabbitMQ) someone must provide an Axon
plugin which looks like a local in-process event-listener
to Axon registered as a local event listener.
This local/in-process event listener can
be provided by Axon out-of-the-box, provided by the MoM
vendor as an Axon plugin, or provided by the application
itself as an Axon plugin. In other words, Axon will have
two main extension-points each declared as an published
Java-interface; one for plugging-in the underlying
persistence engines (SQL or no-SQL) and another for
plugging-in the message listener. All local/in-process
from the stand-point of Axon.
In other words, this model is exactly
the lightweight local transactional event distributor
we are discussing in the other thread, when it comes to
event distribution.
At the end I know many of the above stuff may not be
clear yet. Mainly because at this time we may not have
the same common set of assumptions yet. So, please
don't hesitate to point-out the unclear parts.
Cheers,
Esfand