Resource injection within sagas

Hello,

I think I am missing something related to dependency injection and sagas. As I understood from the documentation, spring related beans are automatically made available using the @AutoWired annotation inside @Saga annotated POJO’s. And they do, at least when the @SagaEventHandler annotated method is being called.

Things go wrong with the @DeadlineHandler method. It gets called as expected, but all injected resources are null. Not that surprisingly off course since they are all transient variables to prevent them from being serialized and stored in the saga repository. But shouldn’t they be injected as soon as the saga is deserialized? Otherwise, a deadline handler would not be able to rely on e.g. an injected mailer service.

I am pretty sure it is just a configuration issue… I tried adding them using injectResources on the SpringResourceInjector available via the AxonConfiguration while creating the DeadlineManager in the spring configuration class, but this does not solve the issue either. One strange thing also, while debugging I noticed the resources I wish to use in the deadline handler are available in the alreadyCreated collection of the Spring resource injector… So it seems there are know the application…

I created the deadline manager using the following code snippet.

@Autowired
private TemplateProcessService templateProcessService;

@Bean
public DeadlineManager deadlineManager(Scheduler scheduler, AxonConfiguration configuration, TransactionManager txManager, Serializer serializer) {
		
	SpringResourceInjector injector = (SpringResourceInjector)configuration.resourceInjector();
    //Adding the templateProcessService here doesn't solve the saga issue either.
    //templateProcessingService is not null at this line.
	injector.injectResources(templateProcessService);
		
	return QuartzDeadlineManager.builder()
	        .scheduler(scheduler)
	        .scopeAwareProvider(new ConfigurationScopeAwareProvider(configuration))
	        .serializer(serializer)
	        .transactionManager(txManager)		        
	        .build();
	}

The thrown exception is

Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'b.t.c.n.e.BalanceExceededSagaHandler': Unsatisfied dependency expressed through field 'templateProcessService'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'b.t.c.n.s.TemplateProcessService' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

For completeness, I am using axon-spring-boot-starter version 4.5.3.; spring boot version 2.5.4.

Again, it has to be something very simple I am overseeing since everything seems to be in place to work as expected.

Thank you for reading my post and providing your feedback.

Kind regards,
Kurt

Hi @KDW,

Can you also share your Saga code?
I suspect something there.

KR,

Hello Lucas,

Thank you for answering. As requested, hereby the entire saga code.

@Saga(sagaStore = "defaultSagaStore")
public class BalanceExceededSagaHandler {
	private static final Logger LOGGER = LogManager.getLogger(BalanceExceededSagaHandler.class);
	
	private static final String NOTIFICATION_SEND_DEADLINE  = "b.t.c.s.notification-send-deadline";
	
	private static final int MAX_RETRIES  = 5;
	

	@Autowired
	private transient TemplateProcessService templateProcessService;

	@Autowired
	private transient TemplateManagementService templateManagementService;
		
	@Autowired
	private transient MailerService mailer;
	
	@Autowired
	private transient WebClient customerQryService;
	
	@Autowired
	private transient DeadlineManager deadlineManager;
	
	
	private BalanceExceededDomainEvent balanceExceededEvent;	
	private int retryCounter;	
	private String accountCreateDeadlineId;
	
	
    //This method will be called as expected with all injected resources set properly.
	@StartSaga
	@SagaEventHandler(associationProperty = "eventId")
	private void onDomainEvent(BalanceExceededDomainEvent balanceExceededEvent) {        
		if(LOGGER.isDebugEnabled()) {
			LOGGER.debug("Balance exceeded for customer id = {} received. Sending notification...", 
					balanceExceededEvent.getCustomerId().getId());
		}
		this.balanceExceededEvent = balanceExceededEvent;
		this.retryCounter = 1;
		this.accountCreateDeadlineId = deadlineManager.schedule(Duration.ofSeconds(15), NOTIFICATION_SEND_DEADLINE);
	}
	
    //This method won't be called, an exception indication unsatisfied resources is thrown
    //before the method body is entered.
	@DeadlineHandler(deadlineName = NOTIFICATION_SEND_DEADLINE)
    private void on() {
        if(LOGGER.isDebugEnabled()) {
			LOGGER.debug("Failed to send notification for customer '{}'. Retrying {}/{}", 
					balanceExceededEvent.getCustomerId().getId(), retryCounter, MAX_RETRIES);
		}
		
		if(retryCounter <= MAX_RETRIES){
			this.sendNotification(balanceExceededEvent);
		}else {
			deadlineManager.cancelSchedule(NOTIFICATION_SEND_DEADLINE, accountCreateDeadlineId);
			SagaLifecycle.end();
			LOGGER.fatal("Tried to send notification for customer '{}' {} times but failed. Stop retrying!", 
					balanceExceededEvent.getCustomerId().getId(), retryCounter);
		}
    }
	
	private void sendNotification(BalanceExceededDomainEvent balanceExceededEvent) {
		try{
			//Retrieve the customer details for the specified customer id.
			Customer customer = retrieveCustomer(balanceExceededEvent.getCustomerId().getId());
			if(LOGGER.isDebugEnabled()) {
				LOGGER.debug("Customer details for id = {} successfully retrieved.", 
						balanceExceededEvent.getCustomerId().getId());
			}
			
			//Retrieve the template to use as notification e-mail message.
			Template template = templateManagementService.findByEventTypeAndEventVersion(
					balanceExceededEvent.getClass().getName(), balanceExceededEvent.getVersion());
			if(LOGGER.isDebugEnabled()) {
				LOGGER.debug("Template for handling events of type {} successfully retrieved.", 
						balanceExceededEvent.getClass().getName());
			}
			
			//Prepare all required template parameters.
			Map<String, Object> parameters = new HashMap<>();			
            //... left out for brevity ...
			
			//Process the template for the given template parameters.
			byte[] result = templateProcessService.process(template, parameters);
			if(LOGGER.isDebugEnabled()) {
				LOGGER.debug("Template processed for {}, version={}. Sending email...", 
						balanceExceededEvent.getClass().getName(), balanceExceededEvent.getVersion());
			}
			
			//Prepare the e-mail message based on the processed template.
			//... left out for brevity ...
			
			//Send the e-mail message.
			mailer.send(destination, subject, body);
			if(LOGGER.isDebugEnabled()) {
				LOGGER.debug("Email message sent for {}, version={}.", 
						balanceExceededEvent.getClass().getName(), balanceExceededEvent.getVersion());
			}
			
			//Notification successfully sent, stop retrying.
			deadlineManager.cancelSchedule(NOTIFICATION_SEND_DEADLINE, accountCreateDeadlineId);
			SagaLifecycle.end();
		}catch(TemplateFailedException ex){
			//Template processing failed, try again...
			LOGGER.warn("Processing the template for the balance exceeded notification "
					+ "failed for customer '{}', cause = {}", 
					balanceExceededEvent.getCustomerId().getId(), ex.getLocalizedMessage());
			
			retryCounter++;
		}catch(MailException ex) {
			//E-mail sending failed, try again...
			LOGGER.warn("Sending the e-mail message for the balance exceeded notification "
					+ "failed for customer '{}', cause = {}", 
					balanceExceededEvent.getCustomerId().getId(), ex.getLocalizedMessage());
			
			retryCounter++;
		}catch(NoCustomerExistsException ex) {
			LOGGER.error("A request to send a balance exceeded notification was received "
					+ "for customer '{}' but the customer does not exist. Stopped trying!", 
					balanceExceededEvent.getCustomerId().getId(), ex.getLocalizedMessage());
			
			deadlineManager.cancelSchedule(NOTIFICATION_SEND_DEADLINE, accountCreateDeadlineId);
			SagaLifecycle.end();
		}catch(NoTemplateExistsException ex) {
			LOGGER.error("A request to send a balance exceeded notification was received "
					+ "for customer '{}' but the template does not exist. Stopped trying!", 
					balanceExceededEvent.getCustomerId().getId(), ex.getLocalizedMessage());
			
			deadlineManager.cancelSchedule(NOTIFICATION_SEND_DEADLINE, accountCreateDeadlineId);
			SagaLifecycle.end();
		}	
	}
	
	private Customer retrieveCustomer(String customerId) throws NoCustomerExistsException {
		Mono<Customer> result = customerQryService				
				.method(HttpMethod.GET)				
				.uri(uriBuilder -> uriBuilder.path("/customer/{customerId}").build(customerId))
				.retrieve()
				.onStatus(
						httpStatus -> httpStatus == HttpStatus.NOT_FOUND, 
						throwable -> Mono.empty()
				)
				.bodyToMono(Customer.class)
		        .retryWhen(Retry.fixedDelay(3, Duration.ofSeconds(2)));
		
		try {
			Customer customer = result.block();
			return customer;
		}catch(Exception ex) {
			throw new NoCustomerExistsException("id = " + customerId + ", cause = " + ex.getLocalizedMessage());
		}
	}

}

The saga store is defined as follows in a Spring @Configuration annotated class.

    @Bean
    public SagaStore<?> defaultSagaStore(EntityManagerProvider provider) {
		JpaSagaStore store = JpaSagaStore.builder()
				.entityManagerProvider(provider)
                .build();
		return store;
    }

Kind regards,
Kurt

A word of explanation with the code above might be appropriate.

What I try to do, while (ab)using sagas for this purpose, is built a mechanism ensuring an e-mail message is always sent exactly once for BalanceExceededDomainEvents with the same eventId. The event id is assigned when the event is constructed making it always unique. The saga should keep on retrying if sending the email message fails for whatever possible reason.

The SagaEventHandler does not do anything else but registering a deadline to be completed within a certain amount of time. Since no other event ends the saga, the deadline handler is always called after the deadline passes (in this example 15s).

When handling the deadline, it will try to process the template en send the mail message. If it succeeds, the saga is ended by calling the lifecycle method and the deadline schedule is cancelled. If it fails, depending on its cause, a retry counter is increased and it will try again after the time-out or cancelled. If the max retry count is exceeded, the attempt to send the email is also cancelled.

Hi Kurt, hope it’s okay if I jump on this too.

I’ve tried to reproduce the issue on my local machine.
So, I had a minimal sample project containing a Saga, with a Deadline Handler and @Autowired annotated resources in the Saga that I made available in the Spring Context.

As you might catch from my wording, all this worked as expected.
Hence I am on the same page as you: it’s likely a simply misconfiguration somewhere.
The question still is what that might be.

For that, I was hoping you might have a small reproducible repository you can point me/us to.
That way, we can debug it locally.
Would you be able to provide something like that with us?

Aside from that, I do want to point out that the ResourceInjector#injectResources method is not intended for external use.
That method is actually used by the SagaRepository to inject the resources into the Saga instance.
The SpringResourceInjector in turn uses the Spring Application Context (through the AutowireCapableBeanFactory) to inject Spring beans into your Saga.

Differently put, simply exposing the TemplateProcessService into the Spring Context should be sufficient for any message handler in an Axon & Spring environment to be picked up. However, that’s what your question is about, so let’s try to solve this!

Hello Steven, thank you for responding.

No problem at all to jump on this too :slightly_smiling_face:

I certainly can share a “small” repository for reproduction. Therefore I took the liberty to sent you an invitation using …:shushing_face:… to enable access the shared repository.

I think your remark that I made available in the Spring Context is the key to the solution. The TemplateProcessService is only annotated with the @Service annotation. I assume this is enough to make it available in the Spring Context… Although I think I might be missing some Spring related aspects…

The use of injectResource felt a bit strange to me as well. Glad you straightened that out!

Awesome, I spotted the invite and accepted it. :slight_smile:
However, it might be me, but I don’t spot any code in there yet…
Is there perhaps anything missing in my credentials over there?

Woops! I think I’ve missed some settings indeed? Hope it works this time…

@Steven_van_Beelen Just a quick check. Were you able to view any code in the repo after updating the credentials?

Sorry, Kurt, I was pretty tasked with other stuff.
I just checked, and yes, I have more credentials.
There’s still no code visible if I move to your project’s page, however.
Just a thought, but wouldn’t a public repo on GitHub be slightly easier in this case?

Hi Steven, no reason to apologize. I am already happy with the fact you are willing to take a look at it :blush:

If one should apologize, it is definitely me for taking this back and forth so many times. I found out I forgot to assign the developer role to your account which I fixed now. If this doesn’t work, I’ll make the repo public.I definitely need to work on my “skills” about SCM solutions :thinking:

Many thanks again and my apologies for the inconvenience. Lets hope I (finally) managed to give you access…

1 Like

I got the code locally, so everything worked out!
I’m going to take a look right now, as I have some spare time.
I’ll keep you posted as soon as I find something.

So, I managed to run the application (after changing the address in the application.properties to localhost) and deduced the following.

I have debugged your app, Axon (both left early), and ended up in Spring world.
I entered the DefaultListableBeanFactory, the getBeanNamesForType method to be exact.

The parameters are given to this method aimed to wire the TemplateProcessService as a field since you have set up the Saga to do field injection. Within the getBeanNamesForType method, Spring constructs a cache Map out of all beans present.

After the construction of this cache, I checked whether it contained your beans.
Sure enough, it did.
However, the key used for storing beans in the cache, namely the Class of the beans, had a different object identifier than the type provided to the getBeanNamesForType.

My hunch is thus that there’s something with your Class Loaders going a miss.
Or maybe a versioning difference between some of your dependencies.
On any note, I can’t see right now how this directly relates to Axon.

As a quick check, I removed the Spring Boot Developer Tools dependency, as that’s notorious for doing “interesting” things with your class loader.
Removing this dependency, however, didn’t solve any of it.

I hope this guides you to look at the problem from a different angle, @KDW.
Let me know whether any of my hunches (versioning differences and potential code that makes class loader adjustments) helps you towards the solution.

FYI, I do feel that Axon should not enter the perpetual exception loop that occurs, by the way.

1 Like

My hunch is thus that there’s something with your Class Loaders going a miss.
Or maybe a versioning difference between some of your dependencies.

Spot on!

Let me know whether any of my hunches (versioning differences and potential code that makes class loader adjustments) helps you towards the solution.

What I did in response to your suggestions. I ensured all pom.xml files of all microservices have the same versions of all declared dependencies and I removed the Spring Boot Developer Tool dependency just to be sure. Finally I removed all existing JAR files in my local Maven repository and did a complete rebuild of all microservices.

It probably won’t be a surprise to hear that this actually did the trick. Even after all those year, I learned a very valuable lesson: ALWAYS clean and build the code if you notice some strange (unexplainable) behaviour. :wink:

Your feedback and look at the code certainly put me in the right direction to solve the problem.
Many many … many thanks for your time and effort!

3 Likes

Great to hear it solved the problem on your end, Kurt!
And sure thing, I was glad to help :slight_smile:

Providing an in-depth sample, as you did here, is invaluable for us to understand how people use Axon.
So, also for me, this was valuable.

Best of luck with the rest of the project, by the way!
And I’m confident we’ll chat here more often.

1 Like