Hello fellow axon users,
for the last month I have been working on my idea of a configurable avro serializer for the axon framework. I had some drawbacks and am far from “feature complete”, but I reached a point where I can share a working example and I’d love to hear your feedback to decide if this is something to explore further and make it an axon framework extension.
Apache Avro is a schema based de/serilization framework available for several programming languages. It is based on json, so an avro message can be treated as a json message. The benefit of avro, and this is what in my opinion makes it a great candidate for axon event store serialization, is that it supports the single object encoding which is a pure binary representation of a message that contains a reference to the schema and the bytes of the message payload with a very low memory footprint.
With regard to event processing/upcasting, we can evaluate the schema compatibility of the writer (publisher) and the reader (processor) of the event. The overall theory might take to long to discuss up front, but I’d love to go into detail if you are interested.
So let’s move to the working example. I created a repository GitHub - holixon/axon-avro-serializer: Provide support for axon event serialization via apache avro. that implements the Serializer interface. It provides an example using a “classic” bank account application running on spring boot and connecting to the axon server and the apicurio open source scheme registry.
The idea is that I define my events schemas up front, register them on apicurio and generate classes using the avro maven plugin (Gradle also available). Then I create a axon application with a configured serializer that leverages the schema registry for deserialization of events by reading the schema ID from the single object encoded bytes and using the schema information to refer to the generated class for that schema.
While working on this, I discovered that to support various registries (apicurio and confluent being the main targets) I need an abstraction layer for the java/avro stuff that is not really related to axon. So in addition to the avro serializer, I created the avro-adapter repository, which itself is inspired by the darwin library . The adapter lib provides a generic api for avro deserialization and a schema registry interface, so the axon serializer will remain agnostic of the concrete implementations and tooling used.
This code is not yet published to maven central, so please be aware, that when you want to run the example application, you will have to clone the avro-adapter repository and install it to your .m2/repository before the axon-aero-serializer extension is able to use it.
If you (I am looking at @Steven_van_Beelen and @allardbz 
) are interested I would love to schedule a web session where I can give you a demo …
Why am I telling you all this? Well, ever since I started working with axon, there was the “xstream” is default, “jackson” is better but “avro would probably best discussion” when it came to event serialization, that kinda froze when Allard told me “we tried it and it didn’t work”. Well, I believe I am onto something here, but before I spend more effort, I really need community and Axoniq feedback. Maybe I am just at a point you already reached and I will fall if I take one more step … or I found a way around the obstacles you already encountered and this could really become something the community or Axoniq itself might want to investigate further. Only one way to find out: Try it, and come back to me.
Really looking forward to hearing from you, so excited
Jan
.