Log stacktrace before creating a DeadLetter

Axon Framework
,
1

Currently,

We can only inspect the type of exception and a message in the dead-letter table.
This is not always sufficient to determine the real problem and where the problem originated.

Therefore, it should be nice that a stacktrace is logged, the first time a DeadLetter is created for a certain aggregate.
This can improve troubleshooting significantly.

Any idea to introduce this in the framework?

2

Well, actually, you should already be able to do that, @Bram.

There is a small caveat that bothers me, and that’s that the SequencedDeadLetterQueue implementations differ with their approach towards logging the cause of a newly enqueued dead-letter.

Here’s the JPA version:

Optional<Cause> optionalCause = letter.cause();
if (optionalCause.isPresent()) {
    logger.info("Adding dead letter with message id [{}] because [{}].", letter.message().getIdentifier(), optionalCause.get());
} else {
    logger.info("Adding dead letter with message id [{}] because the sequence identifier [{}] is already present.",
                letter.message().getIdentifier(), stringSequenceIdentifier);
}

And here is the JDBC version:

if (logger.isDebugEnabled()) {
    Optional<Cause> optionalCause = letter.cause();
    if (optionalCause.isPresent()) {
        logger.info("Adding dead letter with message id [{}] because [{}].",
                    letter.message().getIdentifier(), optionalCause.get());
    } else {
        logger.info(
                "Adding dead letter with message id [{}] because the sequence identifier [{}] is already present.",
                letter.message().getIdentifier(),
                sequenceId);
    }
}

As you see, the approach differ slightly when something’s logged…
Nonetheless, the statement comes from the Cause.
This Cause in turn originates from the EnqueueDecision.

And it is the EnqueueDecision you can control, if you like, by configuring an EnqueuePolicy.
The default implementation does the following:

(letter, cause) -> Decisions.enqueue(ThrowableCause.truncated(cause))

Thus, a default decision to always enqueue a given letter regardless of why it failed.
The cause is, however, truncated, specifically to cater to the fact the cause is stored in the dead_letter_entry table and should not exceed a given size.
FYI, the implementation of the truncated(Throwable) implementation is the following:

public static final int TRUNCATED_MESSAGE_SIZE = 1023;

public static ThrowableCause truncated(Throwable throwable) {
    return truncated(throwable, TRUNCATED_MESSAGE_SIZE);
}

In conclusion, I assume you could already log the reason why a letter is enqueued by adjusting the log level for the SequencedDeadLetterQueue in question.
If the cause is truncated because it’s to long, I would recommend defining another EnqueuePolicy through the EventProcessingConfigurer#registerDeadLetterPolicy operation for a specific processing group or EventProcessingConfigurer#registerDefaultDeadLetterPolicy.

So, I hope this helps you further, @Bram!

3

Thank you for your detailed answer @Steven_van_Beelen!

1 Like