Calling 3rd party API's using events, queries or ...?

[KDW (Kurt)]

What would be the recommended/best practise approach for the following use case?

Assume the use case where you have a Food Service used to retrieve information about a certain types of food based on a description (for the summary) or an id (for details). The summary and details are provided by third party service providers. The number of 3rd party providers can change in the future for whatever possible reason.

The “search” service has the following schematic representation.

A second service, the “provisioning” service is responsible for calling any third party using an external API. This service has the following schematic representation.

I choose to use two different services for this purpose from scalability point of view and to simplify the possibility to add additional 3rd party providers in the future. Third party requests could be handled by the same instance of the provisioning service or multiple instances of the provisioning service for each 3rd party provider.

The search service maintains a local datastore containing all data that has been retrieved via the provisioning service for future use. Due to usage terms, the cached data needs to be removed every 24h so the local store gets cleaned every cache eviction interval.

Whenever a call to the search service’s REST API is made and no requested data exists in the local cache, a SearchFoodEvent is sent to the provisioning service to retrieve data from the external source. If data is found, the search service is notified via a FoodsFoundEvent. The same applies for details about a certain type of food using the RetrieveDetailsEvent and the corresponding FoodDetailsRetrievedEvent.

As a side remark, each time the food service stores/remove a type of food in/from its local database, a corresponding domain event (used by services outside this boundary context) is sent.

I have the following remarks/questions about this approach:

1. Are events a suitable mechanism to trigger the corresponding actions in this use case, or are commands better suited?
   To my understanding, commands signal an intent to update the state in the domain (maintained by an aggregate). Since the actions in this use case are no changes to the state of an aggregate (there is no aggregate involved at all), commands don’t seem a valid choice.

2. The REST API call of the search service might send an event after which it has to wait for the response by the provisioning service. The call should therefore be asynchronous, which is fine, but I would be required to provide some mechanism to wait for the response of the provisioning service to be signalled to the search service.
   This seems to introduce avoidable complexities and doesn’t feel right…

3. Since the described flow looks like some kind of query behaviour, I wonder if queries could be used instead of relying on events to handle the flow?
   Since Axon Framework queries are point-to-point, scatter/gather or subscription based, what would be the best choice? P2P would involve selecting in advance which 3rd provider to use for a certain request since there should only be one query handler in place to handle the query.
   What if you would like the case where any response is fine to proceed or what if you would like to use some cascading/fallback mechanism?
   The same applies for SG-queries where all query handlers need to respond before you can process their results (at least if I understand correctly).

4. Assume you have multiple provisioning services for each individual 3rd party provider. How can you select the appropriate service using the same event definition? A dedicated event property might be a workaround…

I am just wondering what others think a better approach could be assuming you don’t wish to group both the search and provisioning service functionalities in the same µ-svc…

Thank you for your opinion and the time spent to read this (way too long) post

[KDW (Kurt)]

Just an intermediate update.

Since there is not much difference between sending an event to be handled by multiple event listeners and sending a scatter-gather query (which is also handled by multiple query handlers), I have chosen the query approach for the current use case.

In this case I want to have multiple µ-svcs (acting as data provider services) listening for the query and provide a response when possible. The query behaves like a generic “signal” for multiple query handlers, each belonging to a dedicated µ-svc, but not all handlers (of the different µ-svcs) are able to provide an answer. It is up to the handler itself to decide to respond according to its capabilities or to just return an empty result.

This is obviously not an ideal solution. An alternate approach would be the use of dedicated P2P queries for each involved µ-svc. This would on the other hand require the definition of new queries each time an additional µ-svc to provide data is deployed. The more generic approach described above does not require additional queries to be defined and implemented but introduces overhead and an additional responsibility for the different µ-svcs. In the end, this could result in unwanted behavior when forgotten…

The major advantage of this approach is that you can rely on the out-of-the-box CompletableFuture’s of the framework’s query mechanism and don’t have to implement something similar for the event based counterpart.

[vab2048]

To my understanding, commands signal an intent to update the state in the domain (maintained by an aggregate). Since the actions in this use case are no changes to the state of an aggregate (there is no aggregate involved at all), commands don’t seem a valid choice.

I would just say that I don’t think this is true (that a command has to update the state of an aggregate).
Most of them will but I don’t think you need to place this restriction on your whole system.

I would view a command as an instruction to your system to do something (which most of the time will be to update an aggregate but not always).

To use a command without it being routed to an aggregate you just need a @CommandHandler defined (on a @Component somewhere) so Axon Server knows where to route it (if you don’t have a field in the command annotated with @TargetAggregateIdentifier you can use a @RoutingKey).

For calling external services - I don’t see what’s wrong having a @Component with a @CommandHandler that does the relevant logic.

I have a somewhat convoluted example of not routing a command to an aggregate here where there is a @Component with a @CommandHandler. And the command definition itself is here (which uses @RoutingKey). It doesn’t call an external service but might be enough for you to understand how it works.

Another discussion you might find useful is here (although it is slightly tangential to your question of calling 3rd party APIs).

[Steven_van_Beelen]

My two cents: use queries for this, as you already went for. Morphing this communication into events feels awkward, as it’s like talking in past tense to get a reply from somebody.

Furthermore, I agree on the choice of scatter-gather as well.
If you want the flexibility to add and remove new provisioners, using this query ensures you get all N results. You can from there filter the Stream result how you see fit.

Perhaps you are looking for a reply from a specific provisioner, then I’d check the MetaData for the origin. If it’s not present, you’d fall back to the next best.