Is Loose Coupling a Downside of Event-Driven Architecture?

If you know anything about me or Solace, your jaw may drop when I say one of the biggest advantages of event-driven architecture – the loose coupling of applications – has a downside. In fact, I believe this downside is why request/reply interactions still dominate the application landscape, even in situations where using event-driven patterns would be advantageous.

Tight Coupling vs. Loose Coupling in Application Architecture

Request/reply interactions using RESTful APIs entail the tight coupling of applications, i.e. direct connections between them: app A asks app B for information, and app B sends app A the requested information.

In event-driven interactions, applications “publish” events to an event or message broker trusting that the broker will get them where they need to go, and systems that need information “subscribe” to topics indicating their needs, vs. seeking out and directly connecting with the sources it might come from.

That loose coupling of applications is great from a development and system perspective because it eliminates the need to develop, create and maintain direct connections between every pair of applications that need to interact. It’s especially advantageous in fan-in and fan-out scenarios, and in many modern uses cases within trends like big data, hybrid cloud, IoT, machine learning and mobile computing.

But that same loose coupling can make the deployment and management of an event mesh more difficult than it should be.

Why is it so difficult to develop event-driven applications today?

If you’re creating a “source” application that just publishes events, life is pretty dang easy: define the event schema and topic name, pick an API and protocol, and start sending. Things get way more complicated if you’re building a “sink” application that needs to receive events, because you need to find and subscribe to events that meet your needs and use case. The same logic applies to processor applications that receive events, do something of value with it, and publish new events and information of their own.

When I was building event-driven applications in support of the FAA’s transformational SWIM initiative, I came to recognize what I call the “Five W’s” that I found myself continually asking:

• Where do you discover events/topics that meet your needs,
  let alone the schema definition that defines their payload?
• What logical event address (topic) do you subscribe to in order
  to receive just the events you want to do something with?
• Why does a given event exist, i.e. what is its context and purpose?
  And if you can’t figure that out… how do you know what to consume?
• Who do you contact to learn more about events and their context/purpose?
• When will a given event be available or deprecated? Where is it in its lifecycle?

These questions are usually easily answered in the RESTful API space since tooling (such as API Management) provides solutions, but the event-driven space is the wild west in this regard. Yes, this blog post is brought to you by the letter W!

Welcome to the Wild West of Event-Driven Architecture

I’ve spent much of my career building event-driven solutions. As I mentioned above, one was the FAA’s transformational System-Wide Information Management (SWIM) project. SWIM is effectively an API and eventing platform that lets national airspace systems exchange information in real-time. It’s an awesome program and one I am particularly proud of.

One of the biggest challenges we faced was giving consumers a way to discover events; at the time we were dealing with a few thousand distinct events including weather condition updates, flight tracking objects and arrival/departure scheduling. We tried all sorts of service registry products, but they were all geared for synchronous web service interface definitions, not asynchronous events and their associated data schemas, let alone at the volumes we were dealing with. We ended up building our own event catalog and portal, which obviously isn’t optimal, and isn’t even practical for most organizations.

Now, working for Solace, I spend my days talking with some seriously smart folks who are implementing all kinds of applications, event-driven and otherwise, for current and prospective customers. I can’t tell you how frequently they lament how the lack of event-centric tooling slows their progress along the path to being event-driven. Those who’ve already incorporated events into their organization have either spent considerable capital to build homegrown tooling, or get by with tools like Excel, Word or SharePoint to keep track of events across the enterprise. There are four problems with these “solutions”:

1. There’s no consistency between teams within the same organization, much less across organizational boundaries;
2. Artifacts become stale, out of date and there is no way to discover what’s actually happening;
3. Information is fragmented and partitioned into organizational silos; and,
4. Developers have no actionable event “contract” that they can use to generate code.

Essentially, they have captured and documented data that’s used by a few people, and not very effectively. And remember I’m talking about companies who’ve succeeded at becoming event-driven.

Now imagine (if you even need to at this point) that you’re just starting down the path to event-driven. You understand the transformational value it offers, appreciate the value of processing events as they occur, and want to use and reuse events in a variety of ways. As you dig deeper, you start to ask the same questions (the Five W’s) I rattled off earlier, and you don’t find easy answers either. Not good!

Replicating the Easy Implementation of REST in the Event-Driven Arena

For many, that lack of answers shrouds the path to event-driven success in enough mystery that they give up and go back to their comfort zone of creating synchronous, request/reply interactions between applications. Why? Because it’s easy to create request/reply interactions using RESTful HTTP thanks to sophisticated tools like API portals and service libraries that accelerate and automate the development, definition and discovery of services. Unfortunately, in doing so not only do they not meet their objectives, they’ve kicked a critical can down the road – and who could blame them!

Whether you are just starting down the road to event-driven or have been on it for years, the lack of event portals, catalogs and code generation is making life harder than it should be and impeding both adoption and success.

In my next post I’ll dig into that problem by breaking down the foundational tools that have made the development of RESTful applications so easy, and I’ll show how that model can be replicated to accelerate and simplify the migration to event-driven applications and interactions.