EventPublicationFailedException while creating an order entity/event in kafka

Imported from Google Groups
1

Setup- Standalone Kafka instance

  • MySQL database as event store
  • Command service implemented using SpringBoot and AxonFrameowrk Kafka Extension
  • Command Service is deployed on K8S cluster

Test Scenario### JMeter script with 500 concurrent test users firing command over REST- Command Service with single instance - All Successfull

  • Command Service scaled to 2 instances - All Successfull

  • Command Service scaled to 3 or 4 instances - Errros observed for certain requests with exception as below

ERROR[0;39m o.a.c.c.C.[.[.[.[dispatcherServlet].log - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.axonframework.messaging.EventPublicationFailedException: Event publication failed: Exception occurred while committing kafka transaction]

Checked the kafka extenstion source and could see this exception is mapped to ProducerFencedException

Seems its generating duplicate event / transaction ids when command services are horizontally scaled.

Please suggest remidy or alternative to avoid these exceptions.