Easily Publishing Events from the Cloud to Solace PubSub+

Most modern enterprise IT systems use cloud-native services to add functionality or improve capacity, performance, and flexibility, but the integration of such services can be challenging. The myriad of different cloud services in use by enterprise applications and their different message schemas can appear daunting to interoperate. There is also the challenge of ensuring a secure infrastructure with proper authentication, authorization, and encryption. It can be difficult to decide where to start.

Today’s microservices architecture has evolved to fill that gap. By enabling simple, scalable, targeted programs that deliver the interoperability a single enterprise or application requires between two different services is key to digital transformation. Software houses, system integrators, and in enterprise DevOps are quickly and efficiently delivering these solutions without requiring the deep platform specific knowledge of a platform vendor. The RESTful HTTP standard is well suited for communicating between microservices and almost any cloud service and has long been integrated into Solace PubSub+ Event Broker.

Solace is building upon that REST integration to make publishing from the cloud to PubSub+ just that much easier with an example microservice for cloud publishing to PubSub+ and the ability to use OAuth with REST producers. Together, they provide a flexible event broker to enable cloud publishing to PubSub+, the open source microservice to get you started, and a guide to help software experts or partners to create the microservices to hook all your enterprise’s event producing cloud services into the event mesh!

Full integration with Cloud Identity and Access Management Services

PubSub+ Event Broker version 9.13.1 fully enables clients to integrate with modern identity and access management systems such as those in many cloud services for both direct integration and with microservice implementations by delivering OAuth for REST Producers! OAuth/OpenID Connect has been added as a method of authenticating and authorizing REST producer clients connecting to the PubSub+ Event Broker. REST producer clients may present a token in the authorization header using the “Bearer” type as a credential during login.

With full support for cloud authorization and authentication by adding OAuth to Solace’s current suite of authentication and authorization capabilities, the next step is to apply it with microservices to deliver the interoperability!

New Tools to Build Microservices for your EDA

Moving to microservices does have its challenges, but with proper preparation you can be ready with the tools and inter-process communication you need to embrace the microservices architecture. This Microservices Overview Video from Solace helps you understand how to unlock the potential of microservices and serves as a great introduction to The Architect’s Guide to Event-Driven Microservices which covers in depth the relationship between the event driven architecture and microservices to deliver digital transformation. Once you have a good understanding of the relationship of microservices and the event-driven architecture, the strategy behind leveraging microservices with the event mesh, service mesh, event management and API management will come clear as the Unified Microservices Platform reveals how critical the right microservices platform is to enterprise digital transformation.

Perhaps that sounds overwhelming when you just want to connect your cloud service to Solace PubSub+! It really isn’t and provides great context for what you are about to do! Now, let’s dive into the crux of the solution.

How to Publish from Your Cloud Service to PubSub+

To get started with the microservice to publish from a cloud service to PubSub+, you need the following:

1. A PubSub+ event broker to consume the events
2. The cloud service that is publishing events
3. A security system containing the authentication and authorization information for the solution
4. A virtual or physical system to run your microservice
5. The microservice code

PubSub+ Event Broker can be an appliance, software, or as a service. The cloud service is provided by your cloud provider. The cloud providers each provide a version of a security system that are frequently used in concert with their cloud services. Serverless cloud solutions are often chosen to automatically spin up a copy of your microservice when the events are being published. When events are frequent, it may make sense to implement on a physical or virtual server as Kubernetes or a VM in a public or private cloud. Last item you need is the microservice itself. The microservice needs to provide the following components:

• Must be able to pull credentials from the security system to authenticate with the cloud service.
• Must be able to authenticate with PubSub+.
• Must be able to consume events from the cloud service. This may be implemented as either a REST Consumer Push or a REST Consumer Pull depending on the needs of your application and the performance of the cloud solution.
• May need to respond back to the cloud service depending on what the cloud service is and how it is configured. If a response is required, response may depend on a response from PubSub+ or may be as simple as an acknowledgement from the microservice. Again, this depends on the cloud service and its configuration.
• Must be able to reconfigure the event from the cloud service into Solace format, this includes:
  • Generating the Solace HTTP target that maps to http://<host:port>/TOPIC/<Solace TOPIC> where host:port would be configured in the microservice. The <Solace Topic> could be configured in the microservice or derived from metadata from the event
  • Any additional headers for Solace. For example, Content-Type to describe the contents of the event. This could be pre-configured or derived from the event.  You will want to create the Solace-Reply-Wait-Time and Solace-Correlation-ID headers, if you want a REST reply from the broker. Typically, the Reply-Wait-Time would be preconfigured while the Solace-Correlation-ID could be created by the microservice, pre-configured, or derived from message data. Additionally, there is the option to copy event meta data in Solace-User-Properties if you want the event to carry the metadata beyond the microservice
  • The message body. Solace consumes the message body as a single object, while the schema from the cloud service may be more complicated. The goal is to format the message body as a single object with the format expected by the end consumer.
• Must be able to generate an HTTP POST message with the reconfigured target, headers, and message body

Since all microservices to publish from a cloud service to Solace PubSub+ have these components in common, the easiest way to create a new microservice is by changing an existing microservice! So, Solace created one for you!

An Example: Let’s publish data from Google Cloud Platform Pub/Sub to Solace PubSub+

One of the most popular cloud services that customers use to publish to Solace PubSub+ is Google Cloud Platform’s Pub/Sub. Solace has created a Google Cloud Platform Pub/Sub to Solace PubSub+ REST-Based Event Publishing Guide with supporting Python code to enable customers to publish from the cloud to PubSub+! This guide provides you with a complete view of the prerequisites to enable the solution, describes the architecture, discusses the different components of the solution, how to change different components of the solution (e.g. from the new PubSub+ OAUTH for REST Producers solution to client certificate authentication), the quick start to get it operational, and a guide for troubleshooting the solutions! As published, the solution uses Google Secret Manager as the security system and Google Run as the serverless solution to run the microservice. It is leveraging Consumer Push to have Google Cloud Pub/Sub push REST messages to the microservice.

GCP Pub/Sub to Solace PubSub+ REST-Based Event Publishing Guideby SolaceProductsThis guide provides an example of how to use the Solace PubSub+ REST API to stream events from Google Pub/Sub to Solace PubSub+ event brokers.Open on GitHub

This guide not only provides the ability to Publish from Google Pub/Sub to Solace PubSub+ but provides the base code for you to create your own microservice to publish from your cloud service to Solace PubSub+ (subject to Apache 2.0 license)! As a member of Solace’s Development Community, you can even contribute it back to Solace’s GitHub to let other customers use it!

Conclusion

Solace PubSub+ Event Broker version 9.13.1, including OAUTH for REST Producers, and beyond along coupled with the Google Cloud Platform Pub/Sub to Solace PubSub+ REST-Based Event Publishing Guide with supporting Python code provides you with the broker features to enable cloud publishing to PubSub+, the open source microservice to get you started, and a guide to help software experts or partners to create the microservices to hook all your enterprise’s event producing cloud services into the event mesh!

Rob Tomkins

See all posts written by Rob Tomkins