Flow Architecture and the FAA: An Unexpected Event-Driven Leader

Imagine a world where events flow freely from one organization to another—where suppliers, shippers, and stores frictionlessly exchange events about their shared product pipeline. In his book Flow Architectures, James Urquhart imagines such a world in 2036, but today you can get a glimpse of the glories—and the challenges—of flow architecture from an unexpected source: the United States Federal Aviation Administration (FAA).

You see, it’s not just Silicon Valley startups that can benefit from event-driven architecture. Government agencies can also utilize events to make crucial decisions – like those around airspace in the United States. In contrast, events can arrive unsolicited.

Even in isolation from Urquhart’s vision of ubiquitous events, the explosive growth and creative utilization of the FAA’s data shows the potential of a limited implementation of flow architecture.

The Basics of Flow Architecture

Flow architecture, as defined by Urquhart is “networked software integration that is event-driven, loosely coupled, highly adaptable and extensible” both inside and between enterprises. Inside enterprises there has been a gradual but accelerating shift to event-driven architecture (EDA).

The most intriguing portion of flow architecture is events flowing between enterprises. It’s an event-based parallel to the synchronous World Wide Web, which Urquhart calls the “worldwide flow.” The worldwide flow enhances real-time collaboration across enterprise boundaries, without the need for constant polling for updated information and brittle connectivity.

For all the promise, there are many barriers to worldwide flow—thus Urquhart’s 2036 estimated time of arrival. That’s because at this point, “Cross organization integration—especially real-time integration– is a ‘do it yourself’ exercise.” Changing that requires a lot of work, both cultural and technical.

Culture Matters

The first step toward flow architecture is a cultural shift away from the “old way” of doing things. As Urquhart succinctly puts it: “Resist REST.”

A request-reply interaction like REST is most developers’ default choice for internal communications. In a modern distributed architecture, perhaps dozens on microservices are tied together using REST calls to achieve a single business outcome. REST is often the definitive choice when an organization wants to communicate with an outside partner.

Combatting this established mindset means a radical change in philosophy. Rather than simply identifying use cases where an event would be helpful, Urquhart calls for organizations to “adopt an ‘event-first’ strategy.” In other words, rather than justify why a service should be event-based, a developer must argue why it cannot be. In a kinder vein, becoming event-focused also requires hand holding and education. As Marian Wright Edelman says, “You can’t be what you can’t see.” Adopting flow architecture means seeing examples of successful implementations, and access to best practices for getting value.

Standards Matter

As developers get into the flow mindset, they must be assisted by tooling. That’s when the second issue arises: “We can’t quickly create tools to share data in real time because there are no mechanisms to do so,” writes Urquhart.

Creating widely used tooling in turn requires common standards for:

• Sending and receiving events – The world wide web exploded in popularity using the HTTP protocol, which was adopted by web browsers everywhere. To date, sending and receiving events can occur using a wide range of protocols, from JMS to MQTT to vendor specific protocols like IBM MQ or Solace SMF.
• Providing context about an event – When a service requests information and waits for a reply, it has a significant amount of “context”. The requesting application know who it’s asking for information, why it needs the information and what to do with it next. In contrast, events can arrive out of the blue, without the receiving service having any clue about the who, what or why. Events, in some standard way, must provide that missing information.
• Automating discovery and onboarding – The widespread use of API Gateways and management tools has made getting access to information using REST either a completely or semi-automated process. In contrast, finding and accessing suitable events is typically an unstandardized, manual process.

Implementing Flow Architecture at the FAA

Driven by the need to modernize air traffic control, the FAA created an event-driven architecture. The story of that hugely successful transformation has been told before. The event mesh backbone of the FAA SWIM (System Wide Information Management) Infrastructure distributes real time information to FAA systems across the United States.

However, the true achievement is that the events don’t stop at the edge of the FAA. Through secure gateways, external partners of the FAA tap into the flow of events. Airlines and other industry partners that need the SWIM data get it in real-time, without perpetually requesting updates. Whether it’s the availability of gates at an airport, the position data of planes in the sky, or weather for a region, external partners have the latest information without needing to ask. The result is a dizzying array of external parties using the data. It truly shows the power of flow architecture, even in a limited form.

The FAA flow architecture ecosystem (source: FAA)

What can we learn from flow architecture at the FAA?

Given its early implementation of flow architecture, what lessons can be learned from the FAA?

On the surface, the FAA is atypical. It has a monopoly over civilian airspace use in the United States, which is certainly an advantage over a typical private enterprise. If you want to fly in the US, you must comply with the FAA, which makes setting and enforcing standards easier. On the other hand, the FAA shares the sky with both NASA (for space launches and re-entries) and the military, meaning that any information exchange between them needs to be jointly determined. And as aircraft transition to US airspace from other countries, event-based collaboration becomes essential as well. Despite appearing like a dominant monopoly, the FAA needs to compromise frequently.

As we’ll see, the FAA wasn’t immune to the challenges of implementing flow architecture. Indeed, as a front runner, the FAA had to devise numerous workarounds in place of industry-wide standards.

Bridging the Gap Between Operations and Technology

Like many ecosystems, flow architecture wasn’t a natural inclination for architect and developers inside and outside the FAA. To evangelize the new system, the FAA created the SWIM Industry-FAA Team Initiative (SWIFT). (Note that in addition to flow architecture, the FAA also excels at acronym creation).

According to the FAA, SWIFT “bridges the gap between operations and technology, addressing questions and concerns raised regarding data sharing and information services.” Every three months an impressive array of participants share lessons learned from production systems, propose joint implementations with the FAA, and demonstrate pilot projects that show the power of flow architecture. The result has been a sense of community, and a rising awareness of best practices in an event-driven world.

Attendance at the latest SWIFT meeting

Adopting Standards & Using Automation

In the absence of mature industry standards, the FAA has worked hard to reduce the friction of flow architecture.

Sending and receiving events
As the US industry regulator, the FAA mandated JMS as the event communication standard for SWIM, facilitating industry-wide implementation. JMS is a widely adopted standard, but unfortunately can only be used by Java based services, making it less accessible. However, to ease the adoption of SWIM data for those less versed in JMS and Java the FAA distributes quick start kits, which provides sample code and SWIM-specific tooling for connecting to SWIM data.

Providing context about an event
This has been a challenging area for the FAA, especially with flight planning. Consider a flight from New York to Los Angeles, which could interact with dozens of FAA systems. The issue of a distributed system is that multiple systems can produce events at the same time about the same flight, which means that some may be out of date, or less important than others. The FAA helps to address the context challenges through adoption of flight-specific standards, including international standards developed in collaboration with the International Civil Aviation Organization (ICAO). Even more standards are under development.

Automating discovery and onboarding
This has been an area of continued progress for the FAA, moving from a fully manual process to a proposed automated system. Warning: more acronyms lie ahead.

• National Airspace System Enterprise Service Gateway (NESG): The initial secure gateway for SWIM data, which required VPN connectivity and extensive manual onboarding.
• SWIM Cloud Distribution Service (SCDS): Second generation gateway, it eliminated the need for VPN connectivity, but still required several manual steps to establish connectivity. It also doesn’t offer bidirectional communication like the NESG.
• Enhanced Cloud Distribution Service (ESCS): A proposed third generation gateway that would offer automated provisioning with bidirectional communication. If you’re intrigued, an investment decision is targeted for September 2022, with a real deployment in 2023-24.

Summary of different SWIM data access methods

An Unexpected Leader in Software Architecture

Government institutions are typically perceived as technology laggards, but with flow architecture, the FAA in many ways is a leader. Uniquely, the FAA is 1) compelled to share taxpayer-funded information with a wider audience and 2) can use its monopoly power to set standards. As a result, many organizations can learn from the FAA’s experience and follow its lead in establishing a flow-oriented culture and add to its push for industry standards. The first step in that journey: buy James Urquhart’s book. Tell him that I sent you.