Reactive non blocking I/O and thread pool hell avoidance based async scalability in Axon e.g. via Play framework, node.js, akka and netit

Imported from Google Groups
#1

Its been a while since I have been on this group… Trying to learn Play, node.js, akka and netit framework and async scalability.

This is Linked In’s video of why reactive programming without I/O blocking dependency was necessary for them to scale and remove latency from the dependency chain of requests… and blocking I/O wait states.

https://www.youtube.com/watch?v=8z3h4Uv9YbE

Being a big fan of Axon framework, I strongly feel that the core tenets of Axon are awesome backed by this kind of a distributed architecture for eventing, messaging and infinite scalability with design for failure… also the freedom from threading hell and to get to non blocking I/O as described in video above and in links below

https://www.playframework.com/documentation/2.1.x/ThreadPools

http://engineering.linkedin.com/play/play-framework-async-io-without-thread-pool-and-callback-hell

Any ideas on how Axon could get evolved around this non blocking paradigm for DDD/CQRS in massively concurrent event based non thread pool hell and i/o wait independent architecture. Feels like a marriage for future here but I could be wrong…

Cheers!

#2

Sorry one typo… one framework is Netty not netit…

#3

Hi,

welcome back. Thanks for the pointers/links to the presentations. Actually, I had been wondering about the same/similar concepts, mostly triggered by the Java 8 API. Besides the Lamda’s and all the syntactic sugar, there are a few new async API’s to use, such as the ContuableFuture. They make it easier to write async code, just like what happens in the presentation.

One thing that will most likely disappear in Axon 3 (or maybe implemented as a default method on interfaces) is the callback. The CommandBus will return a ContinuableFuture, which allows you to execute logic on the result, once it’s available. Obviously, without waiting for that result in the calling thread.

There are likely more places where things like this can be applied in the framework. If you have any ideas on where (and more importantly: how), please let me know. It will take a while for Axon 3’s API to be “complete”, so all ideas are welcome.

Cheers,

Allard

#4

Cool… Here are some thoughts on the what and how

  • seriously look at the akka Actor heirarchy model (inspired by Erlang) for “let it crash” into Axon core by wrapping Command Handlers, Aggregate roots, Entitty, Sagas and POJO event handler type actors (maybe persisestence actors also) into the underlying core
  • Leverage cluster capability of the akka framework however not visible to the Axon Core API (this would be beautiful if possible)
  • The mailbox pattern for events enabling snapshotting/event sourcing on the Actor subsystem heirarchy gives true reestart and let it crash capability on the actor subsystem. Lot of these type capabilities are core to akka and could simplify Axon further.

Seriously encourage you to deep dive into AKKA
http://doc.akka.io/docs/akka/2.3.5/java.html?_ga=1.209799185.566199763.1409683514

with Netty under the covers and compare the auto scaling and auto healing capabilities of a futuristic Axon with these capabilities vs current Axon. I think Akka could beautiful map to the axxon core under the hood and give this promise of infinite scalability and healing with streaming end point bootsrapping and capability for true async clients for the best performant architecture and with fault tolerance and resiliency at all levels.

I think there needs to be a layer for Command down heirarchies which could be blocking or streaming depending on clients. This is where the REST endpoints or websocket type async channels for commands/queries also can get very interesting with a full DDD/CQRS pattern and some kind of addon play module for Axon maybe. Also bootstrapping Axon without servlets and threading challenges would be very powerful.

I strongly feel that the DDD/CQRS design patterns are so powerful for designing large software systems but this type of scalability would take the solution to the next level. Also, event sourcing may be easier with akka messages under the hood and akka persistence technology.

Also, heavy Spring transacting can be optional or removed for blocking and maybe two modes for Axon where failure for heirarchy (faill all, fail one/restart) vs transactions would be the future direction where transactions are headed. They even have a reactive driver for Mongodb to do async non blocking persistence.

These are some initial thoughts. Will continue to read more… however I think something like this a best of two worlds could be market disruptive and game changing for the enterprise systems space. Almost too good to be true.

I think the Axon abstractions are loose enough for underlying core and starting to build this kind of control hierarchy of actors for Axon and with Event sourcing and immutable events which is core of the reactive strategy and also core to Axon and DDD/CQRS.

Cheers!

#5

I think there needs to be a layer for Command down heirarchies which could be blocking or streaming depending on clients. This is where the REST endpoints or websocket type async channels for commands/queries also can get very interesting with a full DDD/CQRS pattern and some kind of addon play module for Axon maybe. Also bootstrapping Axon without servlets and threading challenges would be very powerful.

On our project, we have a need to apply back pressure to rate limit incoming commands. We haven’t started on the prototyping yet, but we have discussed using RxJava ( http://reactivex.io/ ) with Vert.x for HTTP-based requests, and Netty4 for raw TCP.

I strongly feel that the DDD/CQRS design patterns are so powerful for designing large software systems but this type of scalability would take the solution to the next level. Also, event sourcing may be easier with akka messages under the hood and akka persistence technology.

One thing to be careful with is ‘transaction’ demarcation. We often have multiple events being the result of one Command. I don’t believe there’s a mechanism in place presently to say “all event handlers have handled all the events for this transaction”.

These are some initial thoughts. Will continue to read more… however I think something like this a best of two worlds could be market disruptive and game changing for the enterprise systems space. Almost too good to be true.

I agree much of this reactive programming seems TGTBT right now. :slight_smile:

JAmes