What Makes a Good Asyncronous API? Lessons from REST API Design

In this post, I’m going to walk you through some lessons from REST API design you can use to create good asynchronous APIs, and touch on the need to tie documentation in with REST API documentation and developer portals. In German we like to say “Aller Guten Dinge sind drei” which means “all good things come in threes” so I’ll walk you through three things that make for good asynchronous APIs.

1. Familiar documentation style – concepts are easier to understand if they are presented in familiar terms or in a familiar way.
2. Put consumer needs first – since you want someone to use your asynchronous API, so you should serve their needs.
3. Consistency of terminology – what terms do you use, what do you align our asynchronous API’s channels and messages with.
   

Familiar Documentation Style

Familiarity of documentation is key because developers should be able to view the documentation in the same place as REST APIs, and expect it to look like a REST API and so on.

Fortunately, the AsyncAPI folks have laid a good foundation with the specification itself and all the tooling around it.

The Async API viewer make it easy for developers to look at an AsyncAPI and understand it intuitively if they have a REST API background.

The screen shots above demonstrate what I consider a not-so-great example – a GraphQL API reference on the left and a REST API documentation on the right. That makes the API evangelist’s point – what’s easier to understand if you’ve never used an API before? Which one presents the gentler learning curve? That’s where graphQL in this case doesn’t come out that well in my opinion – I can pick an operation – like “get a pet” from the OpenAPI and try it immediately or I can start learning about queries and objects and mutations to be able to “get a pet”.

On the other hand, the AsyncAPI initiative has done some really great things which makes Async API documentation look so much better, in the viewer it looks very similar to REST APIs because (amongst other things):

• channels that are like resources
• a server section like in OAS
• embedded schemas like in OAS

There are a lot of things that are familiar to you if you’re coming from the REST world. The screenshots below show two examples of developer portals that present and provide access to both API styles and allows developers to gain access to these APIs. You also see that the same representation is used across these developer portals from two different vendors which also make your case stronger – as a developer I can easily find my way on a partner’s or service provider’s developer portal.

Now that we’re in the court and can make our defence, let’s move on to the second thing – “consumer needs first”.

Consumer Needs First

You need to think about the consumer expectations – how can you make your events easily understood? And how can you match developers’ expectations of your APIs to their REST API background.

Group Events: Event APIs not Events

The first aspect you need to consider is the need to group events into event APIs.

Developers think in resources grouped into APIs. They don’t need to understand whatever is behind the interface, its internal intricacies and its internal domain model.  For events, you need to present them with an interface that is easy enough for them to use – developers will not have a good understanding of your internals and even more importantly if you look at events from a purely technical – a broker’s – perspective events are very loosely coupled so you need to document how they are related.

In the traditional messaging-oriented middleware and event broker context events look like as discrete things rather than a part of an interface.

So, to serve the needs of developers familiar with APIs you need to group related events into event APIs, and then manage the life cycle of event APIs instead of (or in addition to) the discrete events. That is what AsyncAPI lets you do.

This example of the messages section of an AsyncAPI declares three different events from an order domain grouped into the orders API. Looking from the outside in it is easy to understand.

Beware the Firehose of Events

There is a danger that comes with grouping events – you could decide to declare that all related events arrive on the same channel. This is an anti-pattern because – as you can see in the AsyncAPI snippet – you now have one channel that can contain any of these messages.

As a developer I’m not in control of what I’ll receive, I’m going to open a hosepipe of events which creates two problems for me

• I’ll need to be able to distinguish payload types from the events I receive and process them accordingly
• I need to filter the events I am interested in from many events I receive.

This runs contrary to how rest developers think – they know exactly which response to expect from a resource, they can apply filters – e.g. query parameters – to tailor the data they receive, and they can get the state of an order directly by specifying its ID as a URL parameter.

Think Resource URLs

So how do you address this? Think about channels in our asynchronous API as resource URLs.

Here’s an example of an order channel. It declares the event type (order, cancelled) and includes some parameters like the order ID as well as parameters in the channel that allow you to filter the event stream.

The example above uses MQTT notation but of course any protocol that is able to express this type of resource URLs and fine grain channels works.

MQTT topic syntax in which a channel (topic) is hierarchically structured using “/” and you can use wildcards for matching channel components. “+” matches any content in a single level in a channel and “#” matches any content in any number of levels at the end of the channel.

This means you can listen for all events for a specific order with the subscription “retail/v2/order/ using wildcards for all the filter parameters (orderType, salesRegion) and for the event type (“cancelled” in this example – so the subscription also matches channels that contain updated or created in this place).

As a second example – if I was tasked to build a dashboard to track all cancelled orders from b2c customers or maybe send an alert if one of these cancellation occurs, I can use this subscription: “retail/v2/order/cancelled/+/bc2-ecommerce/+” again using wildcards for the “orderId” channel parameter – as I am interested in any matching orders – as well as the “salesRegion”

This puts the developer in control of what they will receive.

Consistency

Whenever you’re designing asynchronous APIs you want to be consistent with other asynchronous APIs in terms of the nouns, the channels and the patterns you use to define the channels, how parameters are used, schemas and so on.

There’s another dimension when you look at this from the perspective of a developer who’s used to REST APIs.

A developer might start out using a REST API that gives me access to orders. Then they get a new requirement that asks them to send a push notification to a customer when an order has changed status, maybe to let customers know their order has shipped. Now they need to be able to listen to events and process them and display them to a user. They need to understand the asynchronous API world from their current understanding of the related REST API.

How can you make this developer experience across different API styles consistent?

There are three things to consider:

Reuse

Here’s an example of an Open API on the top and an AsyncAPI on the bottom (there will be more examples in the same format later). The Open API declares a 200 response to someone asking for an order by id. It declares the requestor receive a JSON payload back and the response adheres to a specific schema. These should be reused in the asynchronous – use the same format – JSON – and schema.

You can see the “schema” reference and the “payload” reference in the example point to the same definition. You can also see the OAS response “content” is in JSON format just like the “schemaFormat” in the AsyncAPI.

The developer can now directly transfer their knowledge – and may even be able to reuse code – everything looks familiar to them.

Align terminology

The second thing you need to do is align our terminology. Here’s an example of an open API resource URL. It includes an order ID embedded as a URL parameter. You can get exactly the order you’re interested in.

In AsyncAPI you can align the channel name on the bottom to follow a very similar pattern to the OpenAPI resource URL.

I’ve added a couple of channel parameters because not everything is exactly the same in AsyncAPI, but you can get pretty close to the resource URL, you can use the same verbiage, the same nouns and can declare the parameter “orderId” to match.

Align filtering mechanisms (parameters)

The third thing you can do when aligning terminology is to look at the parameters of the REST resource and the AsyncAPI channel. I already explained at the order id in the previous section, but here I’ll dig a little bit deeper.

The OpenAPI resource at the top can be used to get several orders. It provides developers the ability to filter by parameters – the orderType and the salesRegion. These are defined as type “string” and they’re declared as query parameters of the HTTP request URL.

How do you transfer that to AsyncAPI? Looking at the bottom example notice the same parameter names and data types. If there was an enumeration for salesRegion – east, west, north, south – those should be reused as well.

Conclusion

If you put all that we have discussed together you end up with this nicely presented AsyncAPI documentation that has a lot of similarities with REST APIs. in a structure that is very similar to that of a REST API, operations that look like resources, channel parameters that look like REST API parameters. A REST-like documentation of an interface that describes how to interact with the system (or systems) that sits at the other end of the asynchronous API.

To recap the three good thing that make a good AsyncAPI:

1. You need to be consistent in how you define REST APIs and how we define asynchronous APIs, i.e. reuse or approximate data formats, schemas, parameters, resource URL patterns, etc.
2. You need to put consumer needs first.
3. You need to be consistent in the terms you use to describe your asynchronous API’s channels and messages.