Unifying Synchronous & Asynchronous API Management with WSO2 & Solace

API management is a well-established way of publicizing, governing, and managing synchronous APIs for internal and external consumption across the enterprise, partners, and customers. However, with the increasing popularity of event-driven architecture and asynchronous APIs, more and more enterprises are asking the obvious question: “How can I offer all my APIs within the same developer portal?”

In this article, we will explore how WSO2 and Solace have partnered to create a unified API management experience across the synchronous and asynchronous world. You can find a great introduction to event-driven APIs in microservice architecture with WSO2 by their Associate Director Dakshitha Ratnayake on Github.

First, let us introduce an example use case which we will refer to throughout this article.

Example Use Case: Connected Asset Management

Enterprises that need to maintain significant assets, whether they are stationary assets such as HVAC systems, elevators, escalators, or mobile assets such as trucks, cars, and trains, leverage IoT sensors and actuators, with ‘always-on’ connectivity into enterprise applications for monitoring and management purposes. These installations are typically multi-site and span across multiple regions.

To achieve this multi-site, multi-region connectivity and data distribution in real-time, these asset-owning enterprises typically adopt the concept of an event mesh.

A typical deployment of an event mesh may look like this:

Figure 1: IT Landscape with Enterprise Event Mesh for Connected Assets

Now, let’s consider the following example:

Due to its global nature, AssetCo has contracted various external maintenance companies (MaintenanceCos) to maintain a sub-set of its asset fleet. As part of their digital transformation, AssetCo’s goal is to provide a better customer service by improving mean-time-to-repair (MTTR) and uptime of the assets at a lower operating cost.

In order to furnish MaintenanceCos with the required data, AssetCo introduces an API management layer with the following capabilities:

• Provide a synchronous API to retrieve asset details
• Aggregate maintenance data and provide a synchronous API to retrieve it
• Provide an asynchronous API for alarms, faults, and preventative maintenance actions

Figure 2: IT Landscape with Unified API Management

Integration of WSO2 API Management with Solace PubSub+

When we started the integration of asynchronous APIs provided by Solace PubSub+ Event Brokers with WSO2’s API management solution, we first looked at the re-usable concepts from the RESTful world that are also applicable to event-driven APIs, as well as identified the key differences. Here is a high-level summary:

Let’s look at a few concepts specific to event-driven APIs:

Policies:

Defining policies for event-driven APIs is far more challenging than in the RESTful world. Policies are more differentiated. By operation, for example:

• A subscribe operation may define the quality-of-service as guaranteed, which, in turn requires the Event Broker to queue all events for each subscription in case the application is not receiving them until the total queue size is reached (and then expire the old ones) and replay them on application re-connection.
• For a publish operation, however, the queue size in general should be relatively small as it is determined by the internally subscribing application. Governing the publishing rate, however, is critical to ensure the Event Broker and backend applications are not overloaded.

There are a few obvious exceptions that are equally simple in both the async and sync API worlds, such as policies restricting the number of concurrent connections an application is allowed to open, or the number of total subscriptions to topics the app can register with the Event Broker. But largely, defining policies for event-driven APIs is much more difficult.

Approval & Access Control:

A common pattern in event-driven architecture is to use parameterized topics which are generated by the publishing application at run-time based on the event payload. This allows for smart routing and distribution of event streams through the event mesh to one or many subscribers. It also allows for controlling access using topic patterns at run-time.

In our use case, such a topic schema could look like this: api/{region_id}/{asset_type}/{asset_id}/{event_type}

where:

• region_id: the physical location of the asset
• asset_type: the asset model/make
• asset_id: the unique id of the asset
• event_type: the type of the event payload, e.g. alarm, failure, action

The MaintenanceCos have been contracted to maintain a number of defined assets, for example, all assets in region R1.

Approving a developer application or subscription to an API therefore needs to consider the MaintenanceCo’s contract and restrict the accessible topics to only region_id=’R1’. This restriction needs to be enforced at run-time, by the designated Solace Gateway Broker.

Protocols:

Event-driven APIs should be able to support a wide variety of enterprise-grade, asynchronous protocols for the developer to choose from but also based on the capabilities we want to offer by the API. For example, MQTT 3.1 offers a quality-of-service flag, whereas JMS offers a time-to-live property and the possibility to read from queues rather than topics.

Quality-of-Service (QoS):

In the event-driven API world, there are two basic QoS available, reliable and guaranteed delivery.

In reliable delivery mode, the event is lost if the subscribing application is not up-and-running.

In guaranteed delivery mode, the API Gateway (in this case, the Solace Broker) stores the event until all subscribing applications have received it. This, of course, has capacity implications. How long events are stored typically depends on the life-time value of the event and the SLAs between the publisher and subscriber, i.e. the API publisher and the API consumer. For example, an alarm event’s value is highest at the time of detection, when the resulting action taken can make a real difference (a safety critical event should trigger the immediate shutdown of an asset). A few days later, its value is only historical, useful for after-the-fact analysis.

The Integrated Solution

The image below describes the integrated experience for designing, managing, and consuming Async and Open APIs across the WSO2 API Manager and Solace PubSub+ Platform.

Figure 3: WSO2 + Solace: Unified API Management Experience

Creating & Publishing an Asynchronous API (1 & 2)

1. Designing Asynchronous API Event Services in Solace PubSub+ Event Portal

An API Developer designs the events in the Solace PubSub+ Event Portal Designer with the parameterized topic schema and event payload schemas. They then create a new Event API Product and add the events to be exposed as channels to it, either as a subscribe or a publish operation. In addition, the API Developer creates the OpenAPI spec for the RESTful APIs.

2. Publishing a Sync & Async APIs in WSO2 API Publisher

The API Team takes the AsyncAPI spec as generated from the Event API Product and registers it as an API in the WSO2 Publisher Portal, adds the Solace PubSub+ Gateway Brokers as Environments where this API should be made available, deploys, and publishes it to the WSO2 Developer Portal. They repeat the same workflow for the RESTful API using the OpenAPI spec.

The Developer Experience (3)

1. Discovering published APIs in the WSO2 Developer Portal

The App Developer browses the API Catalog, views the documentation as generated from the AsyncAPI and OpenAPI specs.

2. Adding APIs to a Subscription

The App Developer creates an application and subscribes to both the OpenAPIs and the AsyncAPIs. The system provisions the APIs on the selected Solace PubSub+ Gateway(s), the WSO2 API Gateway and interacting with the WSO2 Key Manager a set of app specific credentials are created.

3. Connecting the End User Async App to the Solace PubSub+ Gateway Broker

The App Developer retrieves their credentials – e.g. username/password or OAuth tokens –  in the Developer Portal, and selects the endpoint provided for their protocol of choice and connects their application to the Solace PubSub+ Broker.

4. Connecting the End User Sync App to the WSO2 API Gateway

The App Developer uses the same set of credentials when connecting their Sync App to the WSO2 API Gateway. The image below shows the data plane of the integrated solution. It uses the WSO2 API Gateway for synchronous communication and the Solace PubSub+ Broker for asynchronous communication. The Key Manager / Identity Service is shared among the two gateways, allowing for end-user app authentication & authorization using a single set of credentials.

Figure 4: WSO2 + Solace: Integrated API Data Plane

Conclusion

The emerging adoption of event-driven architecture across enterprises is driving the need to create a unified experience across synchronous and asynchronous APIs. Before and during the integration of WSO2’s API Management solution and the Solace PubSub+ Platform we have identified a number of common patterns and concepts but also realized where the differences are and how these can be incorporated into existing, synchronous API Management concepts.

Although the integration is by no means complete and covers only the fundamentals, the combination of WSO2 and Solace to provide unified API Management is based on two proven platforms in their own right. At the time of writing, the integrated solution is available as a beta version.

Further work will include a more sophisticated approval workflow and permissioning for every application as well as integration of run-time metrics.

References

AsyncAPI Specification. (n.d.). Retrieved from AsyncAPI.com: https://www.asyncapi.com/docs/specifications/v2.0.0

Forrester. (2021, April 9). Use Event-Driven Architecture In Your Quest For Modern Applications. Retrieved from https://protect-us.mimecast.com/s/SayMC2kr5DSMWqzf1cd4G?domain=reprints2.forrester.com

Ratnayake, D. (n.d.). Event-driven APIs in Microservice Architectures. Retrieved from https://github.com/wso2/reference-architecture/blob/master/event-driven-api-architecture.md