Integrating Google Cloud Pub/Sub with Solace PubSub+ Using Push Subscriptions

In this blog post I’ll explain how to create Google Cloud Pub/Sub Push subscriptions to a Solace PubSub+ event broker using OAuth authentication protocol. The minimum version of Solace PubSub+ which can support Google Cloud Pub/Sub Push web hook subscriptions using OAuth is SolOS 9.13+.

What is Google Pub/Sub?

Google Cloud Pub/Sub is a messaging service provided by Google Cloud Platform, and it’s a popular choice for application integration. It is used for use cases like the ingestion of user interaction and service events, real-time data distribution, and reliable data exchanges. Publishing applications send messages to topics, which are named “resources” in Pub/Sub, associated with a message storage. Clients who want to receive the messages create subscriptions to these topics. Pub/Sub supports two types of subscriptions:

1. Pull subscriptions: The subscribing client initiates a pull request or a streaming pull request to obtain the messages from the topic.
2. Push subscriptions: The Pub/Sub server initiates a request to your client subscriber to deliver messages.

What is Solace PubSub+?

Solace PubSub+ Platform is an event streaming and management platform that helps enterprises design, deploy, and manage event-driven systems that span cloud, on-premises and IoT environments so they can be more real-time. PubSub+ Event Broker has similar concepts of topics and subscriptions. A PubSub+ topic is hierarchical classification for specifying a message destination. (You can learn about the disadvantages of static topics here.) A publishing client sends a message to a topic and a consumer can subscribe to that topic to receive the message. So far, these concepts still look the same, but I’ll explain how they work and differ below.

Why Use Solace PubSub+ and Google Pub/Sub Together?

One of the key use cases for using PubSub+ and Google Pub/Sub together is integrating systems that are running in multiple cloud and on-premises environments, as PubSub+ can be used to build an event mesh that links them, as shown here:

How to Integrate Google Cloud Pub/Sub with Solace PubSub+

Here, I will explain how to connect Google Cloud Pub/Sub with a Solace PubSub+ Event Broker using Cloud Pub/Sub push subscriptions. Using push subscriptions makes the integration process between the two systems real-time and asynchronous. This diagram shows some of the entities that might be involved in such a use case:

A publishing system would typically publish messages to a topic. As mentioned previously, a Pub/Sub topic is a resource which associates a named address with a message storage.

Creating a push subscription instructs the Pub/Sub service to deliver the messages to a specified web hook URL and optionally authenticate via an OAuth token. A subscription is associated to a particular Google Cloud topic, and can be used to deliver the messages from that topic to a Solace topic (or a queue) using the Solace broker’s REST service.

A Solace consumer can retrieve the delivered messages by subscribing to the Solace topic, or by setting up a queue with a subscription to the Solace topic that attracts the messages and stores them for guaranteed delivery. This way, if the consumer is not connected, the messages can be safely stored for delivery. Another option is to use a REST delivery point (RDP) to send the messages from a queue to the consumer via client provided REST URL.

Setting up a Google Cloud Push Subscription

The first step is to create a topic into a Google Cloud Pub/Sub project (an account and a project should be created) like this:

The next step is to create a subscription to the topic, like this:

Then you need to give the subscription a name and choose “Push” delivery type.

The endpoint URL is what the push subscription will call to deliver the messages. Solace broker’s REST service can be invoked with an API endpoint like /gcp/pubsub/pushtopic, which will send the messages to a topic with the same name. You can also use the REST API to send the messages directly to a queue by passing an endpoint string like /QUEUE/gcp/pubsub/receivequeue, but this example will follow best practices for event-driven architecture (EDA) and publish to a topic in order to decouple publishers from consumers.

You also need to enable authentication (unless Solace authentication has been disabled, which is not recommended) and specify a Google service account to be used. The audience claim is optional, but we will set it in this example to solace, otherwise we will have to use the default value provided by the Google OAuth service.

If a message arrives in the topic, Google Cloud Pub/Sub Push subscription will send it to the Solace endpoint using OAuth token attached as an HTTP header. More information about the OAuth token can be obtained by validating it using Google Cloud OAuth token API:

GET https://oauth2.googleapis.com/tokeninfo?id_token=

{
"aud": "solace",
"azp": "115647238076873867319",
"email": "solace-test@sol-psg.iam.gserviceaccount.com",
"email_verified": "true",
"exp": "1657565174",
"iat": "1657561574",
"iss": "https://accounts.google.com",
"sub": "115647238076873867319",
"alg": "RS256",
"kid": "1bd685f5e8fc62d758705c1eb0e8a7524c475975",
"typ": "JWT"
}


Configuring Solace PubSub+ Event Broker

Before you can publish a message, you need to configure the Solace broker to accept the Push subscription requests by first enabling OAuth authentication service:

Then you’ll set up an OAuth Profile, which will be used to validate the OAuth tokens:

The important settings to configure in the OAuth Profile are shown above.

• Use the Audience claim as an OAuth Client ID in the push subscription settings: “solace”. There is no need to set any password.
• The OAuth role should be Client, since the client is the audience for the token provided.
• The Issuer identifier can be set to “https://accounts.google.com” as it can be also found in the decoded token above.
• The Discovery endpoint should be set to “https://accounts.google.com/.well-known/openid-configuration”.

Additionally, some settings are used for the users Solace broker authorization.

Authorization Groups Claim Name indicates what authorization group will be associated with the connecting client and the Username Claim Name is the claim value, which will be used for the authenticated user. In the show example the “email” claim is used for both (using “azp” for a username could be an option, since it is a unique authorized party ID, however, it could be much harder to recognize it than an account service email value):

 Make sure the Required ID Token Type is set to “JWT”:

Next, configure the authentication settings for the Push subscription by setting up a new authorization group with the name of the Google Cloud service account used in the subscription. The value of the account will be issued in the token claim email like this:

The step is to create a queue named gcp/pubsub/receivequeue and add a subscription to the topic gcp/pubsub/pushtopic to which the messages will be delivered:

Sending a Test Message

You can now go to the Google Cloud UI, select the topic created earlier, and publish a test message:

If everything was configured correctly, the Solace broker UI should now show a message spooled in the queue that was configured to attract the messages:

Conclusion

Enterprise integration often includes a messaging backbone for service orchestration. Solace PubSub+ is a feature rich event streaming and management platform that can be used to build an event mesh, and can be deployed in all major clouds top give using Google Cloud services greater integration capabilities. If you want to learn more about integrating Google Cloud Pub/Sub and Solace PubSub+ I suggest you visit our active developer community.