Scalability of Axon

Imported from Google Groups
#1

As I dive deeper into the exploration of this framework, here are some observations and please correct me if I am wrong.

  • The management of JVMs could and should not be done by the framework. Application server scalability with load balancing at the app server tier, database load balancing with data tier and grids of databases (e.g. mongo, google big table, oracle 11G grid, mysql clustering) can handle very advanced write clustering and read too. However a write db could be relational and the read could be search/analytics friendly like Lucene
  • Spring does a good job of giving piece parts to integrate non VM centric frameworks
  • I see the XStream solutions may require Executor type management of multiple VMs on multiple nodes. I think embedding that type fo logic in the framework will raise a red flag for lot of enterprises who rely on standard scaling through horizontal technologies and not app framework/code specific.
  • Other potential solution for eventbus is leveraging Active MQ/Rabbit MQ for guaranteed message delivery to endpoints.
  • I think default spring plug-ability would really be cool in addition to what you have and staying away from managing synching between VMs.
  • Webapp load balancing with session affinity and hardware/software load balancers is another way to consider the load distribution across incoming requests to VMs, From there writes could be farmed out to a db cluster for write and read via persistent queue cluster/bus on something like Active MQ/Rabbit MQ.

Just brainstorming…

Would like to understand your thought and hypothesis to make this solutions JVM aware, rather than a overlay on top of popular technologies like Spring and spring projects, (Integration, batch, spring mvc, spring -ws, Active MQ, Db clustering, Lucene for Read access indexing etc). The adoption of these technolgies is widespread in fortune 500 enterprise. The plug-ability and non proprietary horizontal scaling capabilities of this framework will be a key driver for adoption and migration push of existing and new apps in this direction.

Welcome your thoughts on this subject. This is really good work on CQRS concept in the Java world.

#2

Hi,

the goal of Axon and scalability (those features will be picked up shortly after the 1.0 release) is definitely not to start managing JVMs. All I want to do is support developers (and perhaps operations) in combining the building blocks necessary to achieve it. Some of these building blocks are MQ components. I don’t have the ambition to build something better than ActiveMQ/RabbitMQ/HornetQ/etc. I just want to make it easy to transport events over these mechanisms.

Concerning the scalability using database clusters and session affinity, that’s actually already the way scalability can be achieved right now. Axon doesn’t try to take anything over in that area. Instead, I strive to deliver a set of tools to use to leverage existing frameworks. But that won’t mean that there won’t be any (additional) components that could help solve scalability problems.

What do you mean with XStream and Executor type management? XStream is just a Object to XML mapper. It is a more flexible serialization mechanism than the standard java serialization. And it provides human readable results. And what the Executor in Axon are concerned: Axon never starts a thread on its own. Where threading is expected, an Executor can be configured to allow applications to supply a container managed thread(pool) where necessary. I’ve been around in large enterprise projects just long enough to know the importance thereof.

Does this answer some of your concerns?

Cheers,

Allard

#3

This definitely helps clarify …thanks…my misunderstanding on XStream…I thought it helped serialize XML to a file and if you had multiple nodes you needed some kind of multi node memory mapped file to access…doesn’t make sense based on your clarification. It only serializes but still stores to JPA pooled end point to scale, however there is a file version, which is One JVM only, I guess??

Really like your thinking…one clarification… I am still struggling with why you would need Executor type entry point/pluggability, expectation of threading for framework use for a thread pool if the jvm is container managed at the appserver level…any clarificaion there would help. I have some experience on millions of transaction apps with frameworks and never had a runnable type incoming request without being attached to webserver session affinity (e.g all usage came via appserver servlett type entry point) to keep things simple and scalable by container, connection pools, data sources, jms queues and those type of exits from a jvm and when jvm ran out of gas other jvms would be load balanced to avoid any runnable type code connectivity from outside into the jvm other than the entry and connection points from the jvm. The only thread needed was for the ones spawned from the appserver on a client request.

One thing I think of is that you may need that here for polling on a snapshot creation type thread, however a spring-batch plugin also could be considered to have a dedicated VM to come in and snapshot. Maybe I am still confused.

Cheers

#4

Hi,

some processing just needs to be asynchronous. Well, you should at least have the possibility to do so. To do that, you need a thread. Instead of creating my own thread, I expect an executor to be provided. That way, the app developer can choose whether to add a container managed thread pool or even a direct executor (to execute it in the calling thread). Maximum flexibility.

In other words, the executors aren’t there to achieve scalability. Just to support asynchronous process where wanted. They aren’t used for entry points either. Just a few of the internal mechanisms, such as snapshotting (which is safe to do asynchronously) and maybe some of the event handlers. Most actions, however, run on the thread that the app container assigned to the request.

The JPA version is as concurrency-safe as the underlying storage mechanism. The FileSystemEventStore isn’t scalable and should (if used) be limited to one JVM. In 0.7, some work has been done on a MongoDB based event store. Creating and plugging in your own implementation for whatever backing system is pretty easy.

Cheers,

Allard

#5

Thanks…to further the dialog here is a scenario and see if I can give an example in two async scenarios

  1. Lets say you have published an event call sendingConfirmation email in the register user example in axon-aution-sample, you have 3 days to respond to that email and after 3 days you have a event which can come in externally which can come in with a guarantee to start the check from the querying the AR and create a event of VERIFIED or UNVERIFIEDANDINACTIVATED.

  2. Also you have every few seconds/milleseconds a snapshotting event has to be published and subscribed to for processing. you could have seperate snapshotting jvm or do it within current jvm with a jms event handler message.

For this asynch capability why not leverage standard jms only and a bus like ActiveMQ or other Enterprise plug-ins to manage the activity and all fault tolerance around that processing activity.

e.g. of time and delays in event processing
http://activemq.apache.org/delay-and-schedule-message-delivery.html

This could also standardize via bus any saga time/publish/subscribe nested processing with simple CQRS techniques.

In this cases your factory could inject an ActiveMQ implementation and implement event publlishers/subscriber handlers similar to what we see in the auction example with Activemq.

And this could be a plugin like everything else, however standardizing on servicebus would be key for all async processing vs native framework capability. I think the nService folks who are leading CQRS talks are also doing it in a similar approach.

Maybe you are thinking the same…just wanted to share some concrete scenarios and rationale and your thoughts.

Cheers

#6

To support applications where scalability is a primary concern, this is exactly what I am planning to do. However, scalability is not an issue for 90% of the applications. My primary target for the 1.0 release is that 90%. After that release, I am going to focus on scalability by supplying adapters/connectors for JMS/MQ technologies.

However, the goal with Axon is not to push anyone in any direction. If you want snapshotting to be done in a dedicates JVM, that’s possible. If you want it done synchronously because it allows you to run in the same transaction, that can be done as well. All you need to do is plug the Executor implementation that does the job for you. I don’t want to force a certain design, just support it. There is no one-solution-fits-all. That’s why I always depend on some interface that allows maximum flexibility. In time, I will provide implementations for various scenarios.

Cheers,

Allard