3 Things to Consider When Choosing an iPaaS Solution
The need to easily integrate and orchestrate increasingly diverse IoT devices, mobile devices and cloud-based
At its heart, event-driven integration means enhanced real-time customer engagement and better-informed, predictive decision-making. The Solace PubSub+ Connector for Boomi enables organizations to reap those benefits faster by creating a combined Boomi and Solace experience that’s greater than the sum of its parts. Boomi developers get to stick with the UI they know and love, while getting added benefits through the event-driven Solace platform. For Boomi architects, the connector fits hand-in-glove with a time-tested methodology and comprehensive platform powering your enterprise’s event-driven evolution. In this post, I will go over the following 6-step methodology that will help you get the most out of your event-driven integration investment.
Step 1: Formalize and Evangelize Your EDA Goals
Step 2. Understand Your Event-Driven Integration Foundation
Step 3. Speed Date Pilot Candidates and Pick the Lucky Winner
Step 4. Event-Driven Design
Step 5. Implement a Quick Win
Step 6. Repeat
Architects using synchronous connections like REST and SOAP have long acknowledged the importance of APIs. There’s a reason that tooling like API Management is such a crucial part of the Boomi platform. API Management enables modelling synchronous APIs, establishing service level agreements, regulating security, enforcing SLAs, registering data formats and ensuring that API users are notified of changes.
Despite their importance, traditional development methodologies (and the realities of rushed modern development) typically leave API definitions until late in the development process: after coding is complete, the developer exports the data structure schema and uploads it to API management.
Lately, thought leaders have turned this development process on its head. Instead of a late–in–the–game export, these thought leaders urge enterprises to define APIs at the start of the development cycle, then implement them using coding tools such as Boomi Integrate. This philosophy is called API-first.
Boomi responded by both modifying the platform to allow an API-first methodology and describing it’s use in prominent blog posts such as this one written by Boomi Field CTO Rene Klomp. While you should read the whole post, this is a summary of their approach to API-first development:
Until recently, asynchronous implementations haven’t had the tooling or methodologies necessary for an API-first development process. That is no longer the case when pairing Boomi and Solace. Much like synchronous API management, architects and developers can now develop event-driven, asynchronous APIs and model them using graphical tools provided in Solace PubSub+ Event Portal. The newly released Solace PubSub+ Connector for Boomi can then seamlessly integrate your freshly created APIs into Boomi and use them as the basis for creating processes that follow best-practices for event-driven integration.
The enhanced tooling aligns with a standardized, repeatable methodology tailored for use in event-driven integration, both with organizations new to the concepts of event-driven architecture (EDA), and those well along the journey to a mature EDA implementation.
Before you begin the journey, it’s helpful to review and solidify why you are moving towards event-driven integration with Boomi and Solace. These goals can serve as key performance indicators in your new architecture and provide a helpful list of reminders of why you pursued this initiative in the first place.
Adding Solace to an existing Boomi infrastructure brings:
Adding Boomi to an existing Solace infrastructure brings the ability to:
While event-driven integration provides numerous benefits, enterprises need to exercise care in their evolution towards event-driven maturity. Making sure that your organization is on board, from executive stakeholders down to junior developers, requires thought leadership and persistence. The integration team needs to start thinking about choreography, rather than orchestration. The API teams need to start thinking about event driven APIs, rather than just request/reply. Support personnel need to understand how events move through the enterprise. Leadership needs to grasp the benefits of real-time information flow across multiple clouds.
Fortunately, Boomi and Solace have collaborated on a wide range of materials to help you evangelize an organization. Here are some good ones:
It’s important to note that evangelism is not a one-time activity. Keeping an EDA evolution on track requires a continuing process of education and realignment to goals as the journey continues. As your company onboards more events and starts to see the fruits of your labor, the evangelism shifts from “why are we doing EDA?” over to “check out these events available within our enterprise, how can your group use them?”. An event portal plays a crucial role in that.
The remaining steps in the methodology ensure that your enterprise captures the benefits of event-based integration, while minimizing the uncertainly, resistance and pain associated with any large architectural shift.
While nothing is more exciting than the possibilities of event-driven integration, bringing the concepts of EDA into reality take a firm foundation. While there are multiple options for EDA, this paper assumes that you’ve decided to use the Boomi and Solace product stack. It’s a potent combination. But getting the most of your foundation requires the strengths of each of the components.
Boomi Components:
 |
Integration: Drag and drop UI, data mapping tools, and a comprehensive library of connectors. Coupling this with support for various integration patterns enables you to build any integrations with exceptional speed. |
 |
API Management: Supporting the full lifecycle of synchronous APIs in a hybrid environment. First, configure synchronous APIs and expose their functionality to both internal and external clients, then test and deploy synchronous APIs and enforce contracts and policies with an API gateway. |
 |
Flow: A low code platform for apps that allows everyone to build workflow applications, without needing to be a software developer. Simply drag and drop your workflow as a flow diagram, and the Boomi Flow platform converts this into a fully functioning workflow application. |
 |
Master Data Hub: A cloud-native master data management (MDM) solution that sits at the center of the various data silos within your business and provides easy to implement, scalable, flexible, and secure master data management hub as a service. |
Solace Components
 |
Solace PubSub+ Event Broker: Modern messaging layer that can be deployed across every environment (appliance, on-premises software, cloud-based software, messaging-as-a-service) and component of the distributed enterprise. Event Mesh: A collection of event brokers in different physical locations, woven together that dynamically routes events from one application or connected device to any other— no matter where they are deployed. Whether it’s in legacy on-premises systems or in the public cloud, private cloud, or somewhere in between, an event mesh takes the guesswork out of getting events where they need to be. |
 |
Event Portal: Event management toolset that enables you to create, design, share, and manage various aspects of your event-driven architecture. Event Portal brings to the async world the same functionality that lets enterprises create API-first architectures. |
 |
Cloud management: PubSub+ Event Broker: Cloud is a software-as-a-service (SaaS) platform that gives you access to advanced features like the publish-subscribe messaging pattern, queuing, streaming, replay, and request-reply, while doing all the heavy lifting to dynamically deploy, upgrade, patch, and operate your event broker services. It is engineered to be secure, resilient, and cost-optimized so you can focus on building applications. |
These components interact to create a complete ecosystem for event-driven interaction:
At the heart of the Boomi-Solace relationship is the PubSub+ Connector for Boomi. This connector lives inside the Boomi Integration GUI, both establishing runtime connectivity and letting you carry out common Solace tasks like discovering event definitions without ever leaving the friendly confines of Boomi. This integration creates a better experience for developers and keeps your EDA design vision on track. As part of the PubSub+ Connector for Boomi, there are multiple sample processes available in the Integration Process Library, which in turn link to extensive walk-through documentation. We’ll talk much more about the connector and process library later in Step 5 where we make our designs a reality.
While we are getting acquainted with the Solace and Boomi components, it might be helpful to review the terminology of event-driven integration. We’ll illustrate using a scenario where a fictional taxicab company hires a new driver through the SaaS provider Bamboo. In turn there are three downstream applications that are all interested in a new hire: Slack, Salesforce and ServiceNow.
Note: there are many ways you can implement an event-driven architecture with Solace. However, the typical scenario and best practice is: publish to a topic, read from a queue. Some of us have tattoos that say exactly that. So, to simplify things, that’s what we’ll describe here. There’s much more in the documentation about other use cases.
Event: an event is a business moment where something changes somewhere in the enterprise. Here, the event is the taxi company hiring the driver. The key information about the hiring event is put into the payload (or content) of the event. For instance, the driver’s name, social security number, and address would be in the payload in a JSON or XML format.
Topic: a topic is a hierarchal string that briefly describes the content of the event to make routing more efficient. The topic is meta-information, separate from payload. The format of the string depends on the organization. The topic for this message is: /taxico/hr/bamboo/Driver/hired/v1/349034
Which can be broken down like this:
Subscription: downstream applications indicate what events they are interested in using subscriptions. These subscriptions follow the same format at the topic string, but the true power comes in being able to use wildcards. So if Slack wants to know about all drivers that are hired, it can subscribe to the topic string: /taxico/hr/bamboo/Driver/hired/v1/* where * is the wildcard for “all”
And if SalesForce needs to know about drivers that are both hired, fired and promoted, it can use the subscription: /taxico/hr/bamboo/Driver/*/v1/*
Queue: When the event broker finds a subscription that matches the topic string of the incoming event, it creates a copy and places it on the queue. The queue holds messages until the downstream application is ready to process them. Some remarkable things about queues: they guarantee order, they preserve messages if there is an issue with the application and they serve as a buffer if there is heavy volume for a period of time and the application can’t handle it.
There might be many matching subscriptions, in which case all associated queues get a copy of the message, which makes publishing to a topic and reading from a queue a terrific way to spread information from one source to multiple interested consumers.
No matter how great your foundation, biting off more than you can chew is sure way to sink a promising event-driven integration project. The key is a juicy pilot project that shows off event-driven integration without a big commitment. But with most enterprises featuring hundreds if not thousands of integrations, how do you pick a winner?
One excellent feature of Boomi is that you can view deployed processes at any stage of the lifecycle through the “Deployments” view of the UI. This can give you a sense of how your organization is using Boomi, and which processes would be good candidates for Solace. These are signs that your Boomi process would be a strong candidate for introducing Solace for event-driven integration:
If you’re using Solace PubSub+ Event Portal, then you have an excellent catalog of applications and events moving through your enterprise. Use this as a starting point to locate good candidates event flows. These are signs that Boomi could enhance your existing Solace event integration:
Applications featuring these characteristics (the more the better) should form a short list of potential candidates from which to choose. From there, winnow down further. The best event-driven integration pilot projects deliver a potent combination of:
A quick win that delivers an improved customer experience can breathe optimism into both technical and business teams, relieve anxious stakeholders and provide momentum for additional projects. If you can simultaneously make developers and support personnel’s lives better by alleviating obvious technical pain (brittleness, performance, resiliency) so much the better.
Now that you’ve landed on a single project for a pilot, it’s time to roll up your sleeves and start designing. As mentioned at the beginning, a key part of the event-driven methodology is an API-first philosophy. Prior to PubSub+ Event Portal, there was no API gateway equivalent for events. With the introduction of our event portal, the designs you create here will serve to document your architecture, but also serve as the raw material for the PubSub+ connector as you implement your design in the next section using the Boomi platform and Solace PubSub+ Platform.
Solace PubSub+ Event Portal contains three main entities: events, schemas, and applications. When developing event-driven integrations with Boomi, you can think of the components like this:
 |
An application represents a piece of software that produces and consumes events. For us, that will be a Boomi Integration Process or an external entity like Salesforce or Bamboo. Applications connect to the event broker in an event-driven architecture and communicate with other applications via events. |
 |
An event represents a business moment that can be communicated to interested applications. An application (here a Boomi process or external application) produces an event, containing a payload that matches a schema on a specific topic. |
 |
A schema describes the payload of an event. Producers and consumers of an event can trust that the payload of the event matches the schema definition assigned to that event. Schemas define a type of payload through JSON, XML, Binary, or Text. JSON and XML schemas have content that describes each property of the schema. The content is either in JSON Schema format or XSD/DTD format. |
Going from pilot project choice to a full-fledged design means knowing the details of your applications, events, and schemas. That means asking a lot of questions. In some ways it’s like a high school journalism class: figuring out the 5Ws+H (who, what, when, where, why, and how) of the events flowing through the enterprise. Equally important is documenting all the answers in a repeatable way that developers can use our design-time processes later.
Each of the key questions below flesh out some aspect of the applications, events and schemas. We can then enter them into PubSub+ Event Portal.
The first step is figuring out what information from the source application is available that could be used by the rest of your enterprise. A schema represents what that data looks like. If you’re using a SaaS or other commercial product, there is a well-defined schema available that you can find in its documentation. If it’s an internal application, you might need to dig a little more to find or create a schema representation. Event Portal catalogues schemas for easy access, and allows Boomi to import them when designing processes:
This is also known as the protocol. Luckily for us, Solace and Boomi support many protocols—from event-driven protocols like JMS and MQTT, to REST over HTTP, to application specific ones like database connections with JDBC or just reading from a flat file. The protocol is key for determining the structure of your Boomi processes and picking the right entry from the process library, which will be addressed shortly.
Flexible, one-to-many event distribution is a key benefit of event-driven integration. That means Solace can distribute an event arriving from a single information source to many interested consuming applications. Each of those applications can be defined within Event Portal.
Just because an event starts with a certain data format, doesn’t mean that consuming systems can accept it in that format. That’s especially true for commercial SaaS applications. The first step is adding the destination schema to the Event Portal. This catalogues it and makes it available to the PubSub+ connector at design time. At which point, Boomi’s graphical data transformation tools can convert the data as required.
By using wildcards wisely, the flow of events to a consumer isn’t all or nothing. Some consumers may only be interested in a geographic region. Others may only care about events involving a large amount of money. Fortunately, with event-driven architecture, the event broker can handle the details of event distribution. This feeds into topic string design. As you start your EDA journey, it’s important to pay attention to topic taxonomy – setting up a topic naming convention early on and governing it. A solid taxonomy is probably the most important design investment you will make, so best not to take short cuts here. Having a good taxonomy will help with event routing, and the conventions will become obvious to application developers. For expert advice on this, refer to my colleague Ken Barr’s post on topic best practices. Event Portal calls the combination of a schema and a topic an event. As you input them into Event Portal, you start to build a catalog of events that you can use in this pilot and in future projects.
Boomi’s graphical tooling often comes into play here, as it lets you extract information from the payload and create a topic string with it.
Again, this is the protocol, but this time it is the destination protocol that the consuming application will use. The combination of Boomi and Solace gives you a wide array of supported protocols.
One guiding principle: decouple information sources from information consumers.
A traditional Boomi process receives information, then uses the branch or process route shape to interact with multiple information consumers. Moving to decoupling unbinds information receivers from information senders, with separate Boomi processes for both. PubSub+ Event Bbroker distributes information between them.
Decoupling allows you to reap the benefits of event-driven architecture you evangelized in Step 1, including simplified error handling, preserving data in the case of disaster, allowing free movement of data from on-premises to the cloud, and rapid deployment of innovative solutions. While decoupling is the key, it can take several different forms, depending on what protocol your source and destination applications can use. If it’s an event-driven protocol like JMS or MQTT, then Solace might be the right choice to interact directly within the information source.
In this architecture, an information provider – which could be an internet of things device like a sensor – interacts directly with the event broker. The event broker routes the incoming event and places copies of it on queues for individual Boomi sender processes. The queuing insulates the Boomi processes from sudden rushes of traffic; if a downstream system can’t handle an event at the time, the event broker will hold it. And because each Boomi sender process gets its own copy of the message, it can 1) complete independently, without a dependency on other processes, and 2) implement business logic that is solely related to the end consumer. So, the Boomi sender for Salesforce logic is completely isolated from the logic for ServiceNow. If it’s not an event-driven protocol, then using Boomi to interact directly with the information source is probably the right call.
If the information provider doesn’t use an event-driven protocol like JMS, then connectors within Boomi can liberate information in legacy protocols into events by grabbing the incoming payload, determining the correct topic string, and publishing the event to Solace. In this case the information provider interacts with Boomi first, then Solace.
Finally, you’ve completed the design work. The reward is a top down view through Event Portal of how your event-driven integration will work. That view is invaluable for communicating your vision for the pilot out to the team and re-evangelizing and re-educating your enterprise for your next project.
Instead of the haphazard Visio diagram that lives only on the architect’s laptop and is likely outdated, we now have a living, visual document that developers, architects and support personnel can access.
An outdated Visio diagram crammed with information
A standardized Event Portal diagram
What’s more, we can now dive into each of the entities to get more information about the schemas, applications, and events that make up our architecture. For example, we can see what schema, topic string, and applications interact with the ResourceHired event that was discussed earlier on.
And as we turn to implementation in the next section, the PubSub+ Connector for Boomi seamlessly brings this information into the Boomi UI.
Now that the Async-API first design completed, it’s time to implement the design. Event Portal work continues to be important; you’ll be using it to import event definitions that ensure that the topic strings and data formats that are used match the design. But now the toolset expands to the Cloud Management capabilities of Solace, the Integration component of the Boomi Platform, and the crucial the Solace PubSub+ Connector, which allows them to integrate.
Note, the implementation and configuration instructions here are intentionally high-level. You can get much more information after importing an appropriate process from the process library, which will guide you step-by-step through the process or from more technical blog posts from Mike Hilmen.
To help you get started, Solace and Boomi created a process library leveraging the PubSub+ connector and Boomi Integration. These processes bring together best practices developed through years of shared experience between Solace and Boomi—all you do is fill in the blanks and enhance with your business logic.
Many of these processes feature key technical aspects specifically aimed to meet the business objectives (Step 1) that made event-driven integration appealing in the first place. The key is to match the use case of your application to a use case in the process library.
| Business Goal | Process Library Feature |
| Use Boomi connectivity options to liberate events from legacy systems | Shapes such as the Salesforce connector, database connector and flat file connector |
| Enable operational simplicity agility when adding features, scalability, and global responsiveness | Focuses on decoupling information receivers and consumers |
| Simplify error handling, improves performance when retrying | By default, has persistent backout queue that can store message during retry attempts |
| Pinpoint data routing based on the event’s content | Utilizes Boomi transformation logic to extract out routing logic from payload into the header |
| Support next generation initiatives | Buffers downstream systems (use low latency) |
| Enhance manageability and auditability | Incorporate the use of global tracking IDs that span across Boomi processes |
| Move from batch to real time | Utilizes transaction splitting capabilities of Boomi |
For each process in the Process Library, in addition to the skeleton code, there is extensive documentation in the form of a Google Code Lab. The documentation shows how to configure the Connector, Connector Operation, and the process itself.
This is where the design-first work you did in the earlier step comes into play. The PubSub+ Connector for Boomi integrates with Event Portal, which gives visibility into your event-based architecture. After picking an event, the connector imports the schema and the format of the topic string, then generates a recommended input queue name based on the name of the connector operation.
Even though the process library covers many use cases, you’ll still need to enhance the processes to make them your own. That could include custom data mapping, enriching the event with information from a flat file or database, or running business rules against them.
Now that you’ve created the Boomi process, it’s time to create the underlying Solace infrastructure event services, the queues that your applications will read off, and the subscriptions that will attract messages to the queues. You can create all of these using the Cloud Management console. Again, the code lab guides you through each click you need to create the objects
With the pilot, as you define events and refine the flow of events through your processes, your Event Portal event catalog also takes shape. The initial event catalog serves as a starting point of reusable events which future applications can consume off the event broker queues. Here, the cycle begins again: searching for additional business uses cases that could benefit from event-driven integration. Only this time, you are ready with some lessons learned from the pilot and a handful of events already available on your mesh.
To briefly recap: why would you use the PubSub+ Connector for Boomi? The main reason is that it guides you towards repeatable best-practices to help you get the most out of your event-driven integration investment. It serves as the glue in a combined Boomi/Solace environment, making your design-first artifacts living, breathing entities that your developers can utilize as they make life better for your customers.
Subscribe to Our Blog
Get the latest trends, solutions, and insights into the event-driven future every week.
By submitting this form, you agree to Solace’s privacy policy: solace.com/privacy-policy/
