Managing PubSub+ Event Brokers the IaC Way with Terraform

In today’s DevOps-driven world, managing infrastructure as code (IaC) has become a best practice for maintaining consistency and reliability across environments. Solace PubSub+ Event Broker offer multiple management approaches, including a powerful IaC solution called declarative SEMP (dSEMP) that leverages Terraform.

Why Terraform?

After extensive customer feedback and market analysis, Solace chose Terraform as our IaC solution due to its widespread adoption and robust ecosystem. This integration enables organizations to manage their event broker configurations using the same tools they use for other infrastructure components.

A Practical Guide to Configuration Management

Let’s walk through a common scenario: promoting broker configurations from development to QA environments. This approach ensures consistency and reduces manual configuration errors.

Configure Development Environment

1. Start by configuring your development broker using the WebUI
   
2. Create and test your configurations (queues, subscriptions, etc.) to get the broker the way you want/need it for your new application
3. Validate your setup with actual application testing

*Note* In the below examples I created a new queue new_order, and assigned my desired configuration like owner, queue subscriptions etc. You can find the results of this in the linked gist file.

Generate Terraform Configuration

Once your development configuration is ready:

mkdir terraform/broker_dev
cd terraform/broker_dev

Create an initial provider configuration file:

terraform {
  required_providers {
    solacebroker = {
      source = "SolaceProducts/solacebroker"
      version = "1.0.1" # supply latest version here
    }
  }
}
provider "solacebroker" {
  # Configuration options

Generate the Terraform configuration:

terraform init
terraform generate solacebroker_msg_vpn.<MSG_VPN> <MSG_VPN> <MSG_VPN>.tf \
  --url <HOST_ADDRESS:943> \
  --password <ADMIN_PASSWORD> \
  --username <ADMIN_USERNAME>-admin
# You may have to supply your entire .terraform path to the generate binary

Prepare QA Environment Configuration

The generated configuration needs to be modified for the QA environment:

1. Extract relevant configuration blocks
2. Update environment-specific references
3. Integrate with existing QA terraform configurations

Note: In this process you are doing an intelligent merge of the new objects into your existing higher level environment configuration. Only the objects you wish to move to the new environment should be copied.

For example, a queue configuration might look like:

resource "solacebroker_msg_vpn_queue" "new_order" {
  dead_msg_queue                = "new_order_DMQ"
  egress_enabled               = true
  max_msg_spool_usage         = 6000
  msg_vpn_name                = solacebroker_msg_vpn.qa.msg_vpn_name
  owner                       = "solace-cloud-client"
  queue_name                  = "new_order"

Apply Configuration to QA

Before applying changes:

1. Review the current QA broker state
2. Run terraform plan to preview changes.
3. Execute terraform apply to implement the configuration.
4. Validate the changes in the QA environment.

Best Practices

• Always use version control for your Terraform configurations
• Implement a review process for configuration changes
• Consider automating the promotion process using CI/CD pipelines
• Maintain separate state files for different environments
• Document environment-specific variables and settings

Advantages of the IaC Approach

1. Consistency: Ensures identical configurations across environments
2. Version Control: Track changes and maintain history
3. Automation: Reduces manual intervention and human error
4. Scalability: Easily manage multiple brokers and environments
5. Compliance: Maintain audit trails of configuration changes

Conclusion

Using Terraform with Solace PubSub+ Event Brokers provides a robust, scalable approach to configuration management. This IaC approach enables teams to maintain consistent configurations across environments while following DevOps best practices.

The accompanying diagrams illustrate both the workflow process and the architectural components of the solution. The workflow diagram shows the progression from development configuration through Terraform generation to QA deployment, while the architecture diagram details the interaction between Terraform, dSEMP, and the PubSub+ Event Broker layers.

By following this approach, teams can efficiently manage their event broker configurations while maintaining the reliability and consistency required in enterprise environments.

Review my sample configurations:
Note: if you look closely at the subscription_topic at the bottom of the terraform you will see \u003e this is the unicode encoding for the > wildcard in the topic subscription

Complete HCL from dev broker can be found at https://gist.github.com/Chaymee/db7ee99184a4a52e542b18b8e9be195a