Managing Solace PubSub+ Event Brokers using Infrastructure as Code

Middleware teams and integration architects face several challenges when supporting infrastructure and CICD Pipelines (or organizations in general) as systems scale. You can’t train every team on every API for every piece of infrastructure, nor is it a good idea to let everyone do anything they want to infrastructure the whole company relies on. You also need to give business units a way to customize infrastructure to meet their needs, which requires investment and can introduce bottlenecks that stifle innovation which can drives teams to find “shadow IT” workarounds.

Enter the world of declarative management for infrastructure.

Why do we need declarative management?

With traditional imperative management, you tell the system what you want it to be in a step-by-step process. With declarative management, on the other hand, you tell the system what you want, and it knows how to provide it.

It is a bit like the difference between writing out an instruction manual for making tea versus just telling the computer, “Tea. Earl Grey. Hot.”  In this example, somebody has defined what “hot” means so that the tea doesn’t come out the temperature of the Sun and vaporize the requester. In the case of an event broker, the middleware team would define the parameters and the developers make the requests.

This is where declarative management comes in.  The declarative approach focuses on what the infrastructures target configuration should be – meaning that the developer who needs to deploy or manage something can define the desired state and the system knows what needs to happen to achieve that state.  When it is applied to infrastructure, it is done using Infrastructure as Code (IaC).

IaC alleviates problems by letting the user provision and support the computing infrastructure using code instead of manual processes and settings — operating systems, database connections, storage systems and, yes, event brokers!

This is why Solace has engineered its platform to make it as easy as possible for the people responsible for maintaining an event broker or event mesh to work with the people who are responsible for publishing and subscribing to events.

There are four ways you can manage PubSub+ Event Brokers:

• The traditional command line interface (CLI) approach
• Two Graphical User Interfaces (GUIs): PubSub+ Broker Manager for software and appliances, and PubSub+ Cloud Console takes on some management functions for cloud managed services.
• A RESTful API called Solace Element Management Protocol (SEMP).
• Infrastructure as Code (IaC) via what we call declarative SEMP (dSEMP), which uses Terraform as the IAC tool.

What is Terraform?

Terraform is a popular infrastructure as code (IaC) tool that lets you define both cloud and on-premises resources in human-readable configuration files that you can version, reuse, and share. These config files let you provision and manage all of your infrastructure components – from low-level like compute, storage, and networking resources to high-level like DNS entries and SaaS features – with consistent workflows throughout their lifecycle.

Terraform started as an open-source project supported primarily by Hashicorp, with the majority of the code commits. In 2023 Hashicorp changed its licensing from open source MPL v2 license to non-open source BSL v1, which led to a forked version called OpenTofu. The company has recently been acquired by IBM and, while the future of Terraform is solid, precisely how it will move forward is uncertain. I believe that the good news is that IBM has a strong history supporting hybrid and multi-cloud tooling (i.e. RedHat OpenShift) and there is nothing to say that this won’t continue.

Many of our customers use (or plan to use) Terraform as their IaC tool. For most if not all of them this license type does not impact their use as its goal is to keep other organizations from profiting from the code base.

Solace Terraform Modules and Providers for both self managed and cloud managed services

• Modules simplify configuration and allow the middleware team to set defaults and bounds for specific application teams.  The simplification comes from being able to configure a resource that represents multiple resources, similar to what Wizards do in PubSub+ Manager.  Setting defaults and bounds allow the middleware team to control resource utilization and limit poor behavior – setting the definition of “hot” as described in our tea example. Five modules are provided for each provider including: services, rest_from_consumer, rdp_to_consumer, jndi_client, smf_client.
• Providers convert the modules into dSEMP with restrictions and validation rules.

Why do we need declarative management?

With traditional imperative management, you tell the system what you want it to be in a step-by-step process. With declarative management, on the other hand, you tell the system what you want, and it knows how to provide it.

It is a bit like the difference between writing out an instruction manual for making tea and saying, “Tea. Earl Grey. Hot.”  In this example, somebody has defined what “hot” means so that the tea doesn’t come out the temperature of the Sun and vaporize the requester. In the case of an event broker, the middleware team would define the parameters and the developers make the requests.

Key Benefits for Users of Solace PubSub+ Platform

• For organizations that are already leveraging Terraform, it will fit seamlessly into their existing processes.
• Move software broker services up and down the CI/CD pipeline in concert with the applications that depend on those services
• Allows the code to be managed in a GitHub repository and then shared across the organization.
• Establishes a golden template that others can use for governance,security and maintenance.
• Can keep costs of compute resources (internally and pay as you go) in check by defining the maximum available resources.
• Enables consistent configuration across hybrid cloud environments
• Enable different environments for different use cases that are required to meet regulatory guidelines
• Free up infrastructure administrators so they don’t become bottlenecks as their event brokers and event mesh are used by more and more teams across their organization.

Where do I get it?

You can find the Solace supported providers on the Terraform Registry.

Demo

Rob Tomkins, the principal product manager responsible for dSEMP, has built a great demo to show off dSEMP in use managing a broker queue in an aviation scenario. This demo is sending topics to specific queues, and which the developer can manage using IaC and promote via the CI/CD pipline.