As humans, our inherent desire is to create value for those around us. It doesn’t matter what we’re doing, it’s just what we innately want to do. We all have something valuable to offer, whether it’s goods, services, expertise… we see someone providing for their neighbors, colleagues, or even strangers, and we want to be that person. It feels good, it’s productive, it’s in our nature. In the context of the connected enterprise, I believe that API products provide that value by giving third parties the information they need to create higher value out of existing information. And event API products are the way to provide it in real-time, and in a scalable and efficient manner. I hope by the time you finish reading this blog post, you’ll agree.
API products are groups of related APIs that fulfill some business functions, often in a bounded content. By bundling APIs around a business domain, outcome, line of business, or even geography, API teams can provide third parties singular tools for accessing all the information they need to accomplish a particular goal. These third parties can include separate lines of business within the same enterprise, external partners, or even customers. These API products are typically built using API management toolsets and distributed using consumable and understandable specs, most commonly OpenAPI.
For example, a retail company may have APIs around product inventory, order status, and customer account data. An application developer could leverage this combination of APIs by way of an API product to deliver customers the information they need to manage their orders.
API resources can exist in multiple API products at once. That same order status API and product inventory API, in combination with a shipping API could combine to form a logistics API product for shipping partners. The same product inventory API and customer account data API could be combined with a pricing API to feed into a third-party price comparison tool.
API products can also serve as a unit of entitlement or a single point of access to a grouping of API resources. Instead of granting individual access to each API one-by-one, enterprises can grant access to a complete set of APIs through a single point.
It’s important to think of API products as just that: products. They have product lifecycles and go through versioning and iterations, they have a product roadmap, they have a target market, and all the other things you would associate with any other software product. For these reasons, they should have a dedicated API product manager that is laser-focused on the usage, adoption, monetization (if applicable), and overall lifecycle of the API product.
Event API products are very similar in concept to regular API products. The key difference is that event API Products use real-time data instead data at rest, as is the case with your standard API products. Generally speaking, REST APIs are effective for simple request-reply commands intended for a single consumer. Event-driven APIs, however, are good for one-to-many distribution of time-sensitive information. My colleague Jesse Menning does a great job explaining these differences in the blog AsyncAPI vs. OpenAPI.
In the previous examples, you can see how an event API Product might be useful. While order management might be a good candidate for a REST-based API product (information is only needed as it is being asked for), shipping and logistics gains significant value if information is being shared in real-time. This real-time event streaming could provide customers with updates on shipping status as they happen, give logistics partners a complete view of where packages are in their journey, and feed into AI/ML models to optimize shipping routes. Any use case that requires an up-to-date notification or the ability to look ahead is a great candidate for an event API Product.
Now that we know what event API Products are and what makes them useful, how can you create one? It’s not quite as simple as grabbing a bunch of APIs and bundling them – there’s a process to creating a digital value chain that will be useful and compelling. In our experience, the five-step process goes like this:
Let’s dive deeper on each of these steps.
The process begins with a Discovery phase in which system events are identified and documented. If you look at your IT infrastructure group, they probably have a vast set of technical capabilities offered to others. Databases, file systems, networking, mainframes, message-oriented middleware and event brokers are all potential places to discover enterprise events. These technical capabilities are then provided to the application teams who make use of these IT services to do their work. But interestingly, even some of these applications can have event brokers built within them, making them yet another potential place to discover event streams. And then business processes that leverage these applications can also generate events that are worth discovering.
When we analyze the IT infrastructure, applications, and business processes, there are three primary things we need to discover about the relevant events. We need data schemas to understand what types of data are being transported. We need topic addresses to understand how we can access them. And we need to know what the application interfaces are so we can understand what event streams are being produced and consumed, and how. Discovering these things will give us the information we need to assess the value of our enterprise events.
Traditionally, this is done manually, but Solace has a Discovery agent as a part of PubSub+ Event Portal that will help extract and catalog enterprise events along with all the requisite schemas, topic addresses and more, significantly saving time in the process.
Once we’ve discovered and cataloged our events, the next step involves assessing all system events and identifying the ones that have value. Enterprises generate more digital actions than ever before. Some are simple system events that are important strictly from a record-keeping, system logging perspective, or exist to solve a problem in isolation, but are not of real enterprise concern. In our retail example from before, you can think of user authentications, user interface statuses, or other sorts of log data that have value for their own purposes, but are not useful when it comes to repurposing that event for other means.
Others have actionable value, and these enterprise events are worthy of being exposed across the enterprise via event streams. For instance, other parts of the organization that are concerned with promotions and marketing might be interested in price change events, whereas supply chain partners might be interested in an item shipped or inventory event. Democratizing these event streams for use across the organization and making them more ubiquitous stands to improve integrated systems and ensure that more parts of the organization are in lock-step.
With a list of high-value, share-worthy events in hand, it’s time to design our event API product by bundling these event streams together. When you’re making any kind of product, the intention is to take the requisite pieces and make them greater than the sum of their parts (in other words, create value), and things are no different with event API products. Designing these event API products starts with a focus on the value your customers want.
Start with what problem your customer is trying to solve – what is the higher value use of these event streams in the real world? Who would be interested in or motivated by creating this higher value? Keeping these end-states in mind helps to make your event API product more useful, and more desirable. And by keeping the outcome in mind from the start of design, you increase the likelihood of success for your product.
You should also ensure that your product is easy to pick up and use. How are developers discovering your event API product? Is it being promoted through an API portal? Is there anything making the event API product appear credible? And once they do discover the product, is the documentation complete and easy to understand? Is it easy to get access to the product itself? Is it distributed using a commonly understood spec or standard like AsyncAPI? All of these things will ensure better adoption of your event API product. And as adoption increases, it’s important to collect feedback early and often.
Once your event API product has been designed and created, it’s time to distribute it across your distributed enterprise. Find a home for it where it will be discovered by its intended users, whether on an internal company site or an external-facing developer API portal. Promote your product via your email lists, social media, or developer communities. And as it is used, show the cool and interesting things developers are doing with your event API products to help inspire even more innovation.
Of course over time, business processes change. New events are brought online, new use cases are revealed, or priorities simply shift. Managing the lifecycle of your event API product to keep up with these changes is crucial to finding and maintaining success. It’s a complicated process because there are so many interdependent lifecycles to keep up with.
Think about the implications of one change to an underlying data schema. That schema is being transported by an event, so that event needs to be updated to account for this new schema. That event is transported to an application or even your event API product, so these elements need to make changes to ensure they understand the new schema being brought in. So one change has cascading effects to all of these downstream effects that need to be accounted for. Each of these elements (schema, event, application, event API product) have their own independent lifecycles that need managing, but there is an interdependent lifecycle to keep in mind as well.
Luckily, there is tooling to help manage this. Solace, for instance, offers an event management solution in the PubSub+ Event Portal that helps to track these changes and ensure that new schemas, topic addresses, and more are accounted for.
Importantly, these evolutions and optimizations, especially breaking changes, need to be well communicated and well documented. If you pull the rug out from under a user, they will simply stop using your product. Simple as that.
Developers are incredibly talented at taking APIs and turning them into new and useful applications and tools, and these new creations can streamline business processes, create positive and modern customer experiences, and produce entirely new and unexpected products and outcomes. Bringing this development process from the RESTful API world into the event-driven, asynchronous API world only stands to give developers better tools to produce those same applications and use cases, but with real-time, one-to-many interactions. As we can see, there is incredible value in exposing real-time event data across lines of business and to third parties.
As Solace’s Field CTO, Jonathan helps companies understand how they can capitalize on the use of event-driven architecture to make the most of their microservices, and deploy event-driven applications into platform-as-a-services (PaaS) environments running in cloud and on-prem environments. He is an expert at architecting large-scale, mission critical enterprise systems, with over a decade of experience designing, building and managing them in domains such as air traffic management (FAA), satellite ground systems (GOES-R), and healthcare.
Based on that experience with the practical application of EDA and messaging technologies, and some painful lessons learned along the way, Jonathan conceived and has helped spearhead Solace’s efforts to create powerful new tools that help companies more easily manage enterprise-scale event-driven systems, including the company’s new event management product: PubSub+ Event Portal.
Jonathan is highly regarded as a speaker on the subject of event-driven architecture, having given presentations as part of SpringOne, Kafka Summit, and API Specs conferences. Jonathan holds a BS Computer Science, Florida State University, and in his spare time he enjoys spending time with his family and skiing the world-class slopes of Utah where he lives.[position] => Field CTO [url] => https://solace.com/blog/author/jonathan-schabowsky/ ) )