Dunith Dhanushka is a Solace Scholar and seasoned IT professional with over ten years of experience in architecting, building, and consulting large-scale event-driven systems by utilizing cloud-native microservices architectures.
Event-driven architecture can be seen as a double-edged sword. When implemented right, it can scale applications to virtually unlimited capacity while ensuring high throughput and more reliability. It also allows for new apps/services (for new use cases) to easily be integrated into the architecture to participate and extend overall solution set, by just subscribing to and leveraging the event flows that are already being published by other existing apps. Hence, event-driven architecture has been the architect’s first choice for building systems that demand high scalability, performance, and reliability.
But since event-driven architecture promotes a loosely-coupled, decentralized architecture with many moving parts, it can be challenging to manage the complexity of the application as it evolves. Especially, finding the ownership of events and applications, event schema evolution, architecture documentation, and dependency visualization becomes challenging as more components and people are added to a project.
Solace PubSub+ Event Portal addresses those problems by providing a central place to design, organize, and govern the event-driven architecture components around message brokers and event streaming platforms. This article takes a fictitious use case and walks you through the areas where Event Portal solves the above problems.
Use Case: Order Fulfillment
Picture a large-scale retailer with a growing customer base. They are in the process of implementing an e-commerce store to improve the overall customer experience. The implementation process takes a phased approach where different e-commerce use cases are on-boarded one-by-one.
E-commerce order fulfillment is a use case designed with event-driven architecture in mind. It will be implemented as a collection of event-driven microservices that collaborate by exchanging business events via Solace PubSub+ Event Broker as the central event broker.
John is the solution architect leading the design of the fulfillment use case. John is joined by Matt and Kayla, two developers who will be implementing the
Shipping microservices, respectively.
Planning the Project
John is a seasoned architect who has been working on several large-scale event-driven projects. So, he precisely knows where event-driven architecture excels and needs more attention. John plans to follow a collaborate-design-build-iterate model to implement the fulfillment use case.
Simply put, that model envisions the following.
- John collaborates with the business stakeholders to capture their expectations around the fulfillment use case. E.g., what information is included inside an order? How many orders have to be processed per hour?
- John designs the high-level architecture of the use case to include the microservices, business events being exchanged, external dependencies, and the event broker environment.
- Matt and Kayla collaborate with John over the high-level architecture. They refine it by adding microservice specific information, such as the events being published and subscribed by individual services and the schema of the payload of each event. Once all are satisfied, Matt and Kayla start working on the implementation.
- Whenever John wants to make a design decision, John repeats the same process.
How can PubSub+ Event Portal help?
To facilitate the process described above, the project team decided to use Event Portal as a central place to design and document the fulfillment use case and a consistent way to communicate the changes in the architecture.
Event Portal is a web application that is a product within Solace PubSub+ Platform. It helps developers better manage their event-driven applications around event brokers. Event Portal serves both architects and developers at a very high level as a medium to visualize, discover, and catalog various components in event-driven applications.
Soon after the project is kicked off, John’s team got the accounts for Event Portal where they can log in, create and browse project artifacts, and collaborate while having appropriate levels of read and write permissions.
Throughout the latter part of this article, I’ll explain how John and the team leverage Event Portal’s capabilities to manage the fulfillment use case.
Step 1: Architectural design
As the first step of the process, John logs in to Event Portal and starts designing various components of the fulfillment architecture. For that, he uses the integrated Designer.
An application domain is a bounded context for a particular business domain. John starts by creating the
E-Commerce Fulfillment application domain to group all related project artifacts.
Events and schemas
Once the application domain is created, the business events in the domain can be defined. An event contains a corresponding schema and version. Also, each event contains a topic address field which represents the topic that the event is published to by the applications at runtime.
John creates the
OrderCreated event to represent the order information originating from the
OrderUI web application. Similarly, the
ShipmentCreated events are also created.
Event Portal also allows for defining payload schemas for the events. In our use case, all events should be JSON formatted.
Applications represent the individual processes that publish or subscribe to the business events defined above. They can be web applications, APIs, microservices, or pretty much any application runtime.
In our use case, John defines two applications to represent the
Orders services and the
Shipping service. At this point, the applications include very high-level details. They will be completed when implementation-specific details are added.
Step 2: Building the Applications
Once the design is finalized, John invites his team members, Matt and Kayla, to review what he has done and refine the high-level design with implementation-specific details.
The implementation boils down to two Spring Boot microservices as follows.
OrderService: Consumes the
OrderCreatedevent, validates the order, and produces either
OrderRejectedevents based on the result.
Shipping service:Consumes the
OrderConfirmedevent, creates a shipment for the order, and produces the
Matt owns the implementation of the
OrderService, while Kayla owns the
Matt first logs in to Event Portal and uses the graph view to understand the inter-dependencies of the components. He also uses the event catalog to browse for events his application should subscribe to and publish to the event broker.
Once he understands the bigger picture of how everything works together, he configures the
- Subscribe to the
- Publish the
Up to this point, everything is in the design stage. Now Matt wants to take the design to the implementation level. How is that possible?
Event Portal allows generating the corresponding AsyncAPI specification for a given application. The generated AsyncAPI specification defines the event interface for the application. It contains the events and schema definitions, channels/topics to which the events are published, subscriptions by which the events are consumed, and other event broker details.
Matt then uses that specification file to generate the corresponding Spring Boot microservice. Once it is done, further business logic related to order validation can be added.
The tooling with the AsyncAPI specification increases the developer productivity by directly generating the backend service implementation. That eliminates the slow, error-prone manual coding. Currently, the code generation supports multiple languages, including Java, NodeJS, Python, etc.
In the meantime, Kayla also logs into the Portal and follows the same process as Matt did. She configures the
ShippingService application to:
- Subscribe to the
- Publish the
Once the code for both services is generated, Matt and Kayla continue to add business logic and test them as needed.
Propagating Changes in the Architecture
The previous sections described how Event Portal could be instrumental in executing an event-driven project with a top-down approach: define the architecture, break it down into components, and implement them. So far, so good.
But enterprise systems can’t stay in their rigid forms forever. New business requirements come in, which expects the architecture to adapt with a minimum business impact. That is where most of the event-driven architecture projects start the downfall.
The ability to detect the downstream dependencies among components saves a lot of time when it comes to planning and executing architectural changes. The dependency visualization provided by event Portal helps detect dependencies among applications components and notify the teams who own those dependencies about the incoming changes.
Releasing a New Version of the OrderCreated Event
Let’s assume there’s a requirement to add a new field
sales_region to the
OrderCreated event. The business requirement is to calculate sales figures against each sales region.
John wants to update the existing event schema to accommodate this new field and communicate it to the applications who have subscribed to the
Fortunately, Event Portal serves as a one-stop shop for this. It allows John to create a new revision for the schema with the sales_region field and attach it to the event.
Then, John finds out that the OrderService will be impacted by this change and informs Matt to use the latest version of
OrderCreated in the
Matt acknowledges the change request, applies the change and makes a new release.
Extending the Architecture: Adding the AnalyticsService
With the fulfillment use case completed and deployed in production, the business wants to measure the sales KPIs to get more insights and identify the areas that need further attention. John is asked to capture all business events and feed them into a BI system. How does he plan to do that?
As a seasoned architect, he thinks of reusing the current business events in the fulfillment use case. So he logs into Event Portal and discovers all the events using its event catalog.
Once events are identified, John creates a new application called AnalyticsService configures it to subscribe to the following events.
John assigns Anita as the owner for the analytics service, as she will own the service implementation in the future.
Hopefully this example has shed light on how PubSub+ Event Portal can help you design, collaborate, implement, and govern a large-scale event-driven system at scale.
Event Portal’s strength is the areas where event-driven architecture requires more manual, laborious tasks to document itself. For example, cataloging and versioning are crucial for clear ownership of events and well-documented payload schemas. The graph view will be helpful to identify dependencies among application components before making an architectural change. Also, having a user access level with proper role-based access controls (RBAC) allows team members to collaborate within the proper scope.
If you are planning an event-driven project around the Solace event broker, I highly recommend using Event Portal as the management plane.