8 More Ways PubSub+ Event Portal can Make it Easier to Manage your Event-Driven Architecture

As you may have seen from my previous blog posts, I have been using a fictitious New York City taxi company (NYC Modern Taxi) to demonstrate characteristics and challenges of a real-life enterprise struggling to integrate real-time data and mobile technologies into their business. In the first I talked about distributed tracing with OpenTelemetry, and in my most recent I explained the value of event enabling Salesforce.

NYC Modern Taxi has decided that implementing their new rideshare app (to compete with Uber and Lyft) with event-driven architecture (EDA) will offer a better customer experience, cost less, and allow them to use event routing. Knowing that they would need some help with their event-driven architecture journey, NYC Modern Taxi began using PubSub+ Event Portal to visualize, govern, manage, discover, and catalog events.

While NYC Modern Taxi knew about the major features of PubSub+ Event Portal from videos and blogs posts, there were still some features they discovered along the way that made innovation easier. Here are eight things you can do with PubSub+ Event Portal that can really make an impact as you implement event-driven architecture across your enterprise:

1. Use tags to understand your event-driven architecture
2. Zoom in for a closer look with fine-grained search
3. Assign ownership to applications
4. Figure out what went wrong with revisions
5. Create deep links and direct links in description fields
6. Streamline Kafka discovery imports using search
7. Export AsyncAPI definitions for an application
8. Reassign objects to different domains

1. Use Tags to Understand your Event-Driven Architecture

A large enterprise with mature event-driven architecture can have hundreds of events and applications. NYC Modern Taxi has a relatively simple system with about a dozen applications across three production domains and one testing domain, but understanding event flows even at that limited scale can still be a challenge, especially with the one-to-many nature of events.

The architects at NYC Modern Taxi decided early on to divide applications into functionality-based domains:

• Operations (which focuses on real-time events)
• Back office (software as a service interactions)
• Analytics (bridges to streaming solutions like Kafka, and tools that leverage it)

Figure 1: High-level overview of NYC Modern Taxi’s architecture

Here’s a glimpse at the Operations domain, which houses applications and events related to the real-time interaction between drivers and passengers:

Figure 2: A deeper dive into the Operation domain

Fortunately, NYC Modern Taxi uses several PubSub+ Event Portal features to reduce complexity by focusing in on relevant information. The first is tags, which let you apply one or many free-text labels to schemas, events, and applications. It’s like how you would label an e-mail as “Taxes” or “Vandelay Industries Partnership” within Gmail. The flexibility of tags means that organizations can use them in many ways, but NYC Modern Taxi uses them to identify use cases such as “mobile application,” “payroll processing,” and “driver onboarding.”

Use case tags give you a great way to make things more orderly. First, just identifying which use cases are associated with an event can be helpful. For instance, the RideUpdated event contains updates on the status of a particular ride – a crucial event for a ride-sharing company.

There are many consumers and multiple use cases to consider when making changes. Tags make it easy.

Figure 3: Click on the ellipses icon for an event, then click on “Tags”

Figure 4: PubSub+ Event Portal shows all the tags for the event, along with the ability to add more

Being able to view tags is great, but for even more insight you can filter the graphical designer view to highlight only the objects involved with a certain use case. In the Designer image below, for instance, only objects involved with the mobile use case are highlighted.

Figure 5: Highlighting events and applications from the Mobile use case

2. Zoom in for a Closer Look with Fine-Grained Search

NYC Modern Taxi attaches labels to events, schemas, and applications, but sometimes you need to look inside an object. NYC Modern Taxi assigns each of its drivers a unique identifier called driver_id and include it in multiple schemas because many of their applications care who is behind the wheel.

Tracking down all those schemas that contain driver_id would be difficult without a centralized repository and a specialized tool, but fortunately for them, PubSub+ Event Portal Catalog has both. Its powerful search tools show exactly which schemas use driver_id and how they use it—whether it’s in topic strings, schema bodies, descriptions, or object names.

Figure 6: Fine-grained search

3. Assign Ownership to Applications

Event-driven architecture allows NYC Modern Taxi to loosely couple their applications, which provides numerous run-time advantages like independent scaling under load, faster response times through parallel execution and simplified error handling. During development, loose coupling lends itself delegating applications amongst multiple independent development teams, which can then create innovative features without needing to wait around for one monolithic release.

But even in a loosely coupled environment, teams still need to communicate, because events and their underlying data structures serve as the interface between applications. If the schema for an event changes, affected teams need to know. If it’s a new field, the other teams may be able to use it to improve functionality. And if the event change isn’t backwards compatible everyone who produces or consumes it better be aware.

NYC Modern Taxi promotes awareness and communication by assigning an owner to each application. That way if one application team changes an event, they can notify affected applications owners in time for them to react. Event Portal furthers that effort by making it easy assign one or more owners to an application (or a schema or an event), so there’s no excuse not to reach out.

Figure 7: Assigning an owner to an application

While the owner is visible on every object, NYC Modern Taxi can also easily track down all the schemas, events, and applications one person owns using a search.

Figure 8: Searching for the objects one person owns

4. Figure Out What Went Wrong with Revisions

Even with the best of intentions and clear lines of responsibility, sometimes things break. As a result, understanding NYC Modern Taxi’s architecture and how it has changed is crucial – especially at 2 a.m. when drivers suddenly can’t log into the app. PubSub+ Event Portal comes to the rescue again!

In a complex environment, changes can have unintended consequences. Figuring out what has changed, when it changed, why it was changed, and who changed it is a crucial step in resolving things. PubSub+ Event Portal keeps a running revision history for all objects, including when the revision was made, who made it, and the content of the schema at the time.

Figure 9: Revision history for the RideUpdated schema. I’m not saying that Marc broke the system (he’s usually very responsible), but maybe give him a call?

PubSub+ Event Portal also lets NYC Modern Taxi support staff compare versions (for instance the current version of RideUpdated that destroyed everything and the previous version which worked fine), and to clarify what is different now and perhaps give a hint as to what’s wrong.

Figure 10: Options for comparing schemas

Unfortunately, even after extensive investigation, sometimes the best course of action is reverting to a version that you know worked. With PubSub+ Event Portal, you can do that with a single click.

Figure 11: Restoring a previous version- a (sometimes) necessary evil

5. Create Deep Links and Direct Links in Description Fields

Like many enterprises, NYC Modern Taxi has many tools in its ecosystem. Luckily, PubSub+ Event Portal’s object description field supports HTML capabilities. NYC Modern Taxi uses those capabilities to create deep links to GitHub repositories and direct links to source processes within Boomi.

Figure 12: Direct link to the Boomi Process

Figure 13

6. Streamline Kafka Discovery Imports Using Search

NYC Modern Taxi sends all its events to Kafka for analytics. To keep everything straight, they use PubSub+ Event Portal to manage their Kafka consumer groups, connectors, topics, and schemas through the Discovery tool. When importing Kafka objects, PubSub+ Event Portal’s search capabilities allow them to dynamically filter the list of imported topics so engineers can swiftly assign them to the proper domain.

Figure 14: Filtering for Operations-related topic strings

Figure 15: Assigning all the topic string to a domain in one fell swoop

7. Export AsyncAPI Definitions for Applications

Even though this one is a little more widely known, it’s still a great tip that NYC Modern Taxi uses to power its Spring Cloud Streams applications (which can you can build for yourself). Once you have defined an architecture, PubSub+ Event Portal can export an AsyncAPI definition for the application with a single click.

Figure 14: Select an application, then click on the ellipsis icon

Figure 16: Click on the Async API icon

You can download either a YAML or JSON version of the AsyncAPI definition, which can be used to jumpstart development with Spring Cloud Streams.

8. Reassign objects to different domains

Like most organizations, NYC Modern Taxi regularly restructures its organization and its technology. PubSub+ Event Portal makes it easy to reassign objects to different domains to reflect the organizational reality using the Move functionality.

Figure 17: Just select an object, then click on the ellipses icon

Figure 18: Then click on move and choose a new domain

The Importance of an Event Portal to Event-Driven Architecture

If you’ve decided to adopt event-driven architecture, I can’t stress enough how much easier PubSub+ Event Portal will make your life. Giving your entire team the ability to collaboratively design events, catalog them for reuse across your system, and actively monitor and manage both their runtime and lifecycle over time is very powerful.

Whether you’re just starting your journey to event-driven architecture or are well on your way, and whether you’re just learning about PubSub+ Event Portal or are in the admittedly fictitious shoes of NYC Modern Taxi, I hope this post has given you a better idea of how a full-featured event portal like PubSub+ Event Portal can make it much easier to achieve and maintain the advantages of event-driven architecture.