An Architectural Look at Managed Subscriptions in Solace

When architecting an enterprise messaging system, there are options surrounding how application subscriptions are added to the Solace message routers. One of these options is the use of a subscription manager. In this article, we will explore what a subscription manager is and look at the considerations surrounding using one. In a future post, I will also walk you through a simple example of how to code a basic subscription manager and its corresponding client application.

What is a subscription manager?

A subscription manager is a custom coded program running in the back end of your enterprise which manages subscriptions for other clients. Solace enables a centralized subscription management architecture by providing a special feature which allows certain clients to subscribe and unsubscribe to topics on behalf of other clients.

In order for an application to be a subscription manager, it must connect to the router using a client username which has been designated as such. Such programs are often referred to as “OBO (On-Behalf-Of) agents”, or “OBO subscription managers”.

The “Managing Topic Subscriptions on Behalf of Other Clients” section of the Solace Java C NET Messaging APIs Guide document overviews the mechanics of how this is done.

A typical implementation of an OBO subscription manager requires client applications to send a request to the OBO agent asking for a subscription. While the client is sending this request to the OBO agent through the router, it is not asking the router for the subscription directly. Instead, the OBO subscription manager validates the user credentials and entitlements using whatever mechanism it needs to, and makes the subscription for the client if appropriate.

Here’s the sequence:

Why use a subscription manager?

The question remains: why would you want a central program to make subscriptions on behalf of others? What is the point of doing this? There are a number of compelling arguments, so let’s dive into them one by one.

Existing external permission systems

Perhaps you already have an existing infrastructure in place that defines entitlements for all of your users. This could be a vendor supplied solution such as Thompson-Reuters DACS or Bloomberg’s EMRS, or a homegrown solution.

One option would be to synchronize those permissions into the Solace router. A future blog post will discuss this scenario in more detail, as there are some use cases where this approach makes the most sense. However, one option which is often preferred is an OBO subscription manager.

At runtime, client applications can send a request/reply message to the OBO agent requesting specific topics. The OBO agent can then validate entitlement against the external data source, and make the subscription on behalf of the client. Changes made in the external entitlement system are seen immediately by the agent. In contrast, a periodic synchronization process likely results in lags between the time data is changed in the external system and the time it ends up being respected on the message bus.

Multiple entitlement systems

This architecture can streamline application development. In enterprises where multiple external entitlement systems are in place, applications would normally have to be coded to these multiple entitlement systems. Applications which are currently using only one entitlement system would have to go through a painful port to be able to use another.

By using a centralized OBO subscription manager, all enterprise applications can be coded to a single interface: the OBO subscription manager. The OBO subscription manager is the one that actually deals with the external system’s entitlement system, encapsulating this knowledge and shielding application programs from it. This increases application portability.

Access Control List (ACL) limits

An OBO Subscription manager can complement Solace ACLs. If you are not already familiar with Access Control Lists, please read the “Managing Access Control Lists” section of the Solace TR Feature Provisioning guide. Note that a very common usage of ACLs is to have a setting of ‘disallow’ for the subscribe topic default action, and then a white-list of exceptions, namely the topics which the profile can access. Another way to say that is “the client can’t subscribe to anything except the named topics”. This list of allowed topics are called ACL exceptions.

The Solace router has an upper limit on the number of ACL exceptions allowed. No problem! An OBO subscription manager can augment the solution to provide virtually unlimited ACL exceptions on the fly! At runtime, client applications could send a request/reply message to the OBO agent requesting specific topics. The OBO agent would then validate entitlement against the external data source, and make the subscription on behalf of the client if appropriate. Note the magic here: the OBO agent can apply topic subscriptions to clients even when that client’s ACL says it lack permissions to do so!

In situations such as this, administrators typically leave clients with a very bare-bones ACL which allows little more than the ability to send messages to the subscription manager. This approach uses very little ACL exceptions, thereby offering a big scaling advantage to the solution.

Topic abstraction and encapsulation

Perhaps your application’s business model is such that each application’s set of subscriptions change quite frequently. Perhaps events that occur trigger changes to some applications subscription sets. An example is an FX trading application. Such cases can be cumbersome to manage on an enterprise scale if the burden of tracking and managing which topics each application should subscribe to is pushed completely out to all the clients. Imagine that you have hundreds of unique applications. With all of them tracking their own states persistently, responsibility is scattered and auditing can be quite difficult. Additionally, there may be policy concerns which mandate this information to not be visible to clients.

An OBO agent can be used as the intermediary to shield the applications from their dynamically changing topic lists. In this case, applications could send a request/reply message to the OBO agent, essentially saying “can I have all of my stuff now”. The agent would interact with the external system to determine what those topics currently are, and make the subscriptions for the client.

A variation on this theme is topic abstraction and redirection. In this way, the OBO agent can act similar to JMS based JNDI topics. That is, an application might have a single conceptual topic alias, while the real physical topic name is only known to the OBO agent. The OBO agent provides the redirection service of translating the abstract logical topic name into a real physical topic.

Auditability

A common requirement in many enterprises is to audit which applications have accessed which data. This might be for capacity planning, fee liability, security or some other purpose. There are a number of ways to do this, including scraping appliance logs, listening to appliance events, or requiring applications to audit themselves. Each of these approaches have shortcomings. A simple way of producing effective and complete audit trails is via an OBO subscription manager. Since all client subscriptions go through the agent, it creates a single, custom coded place where you can easily log out all subscriptions made.

Considerations when using a subscription manager

Capacity Planning and Scaling

Without an OBO subscription manager, client applications make subscription requests directly against the router. In that case, you are not faced with any additional capacity planning or scaling requirements than you already faced in your selection of Solace hardware or VMR solutions. However, if you introduce an OBO subscription manager into your architecture, that OBO subscription manager will need a place to run. This implies external hardware and software and all of the capacity planning requirements that come with it.

You will most likely need to consider how you can scale up. Typically this is achieved by deploying a cluster of OBO subscription managers. To achieve this with direct messaging, client applications are coded to set the DTO flag on messages (see “Deliver to One Subscriber” in Solace Messaging APIs Developer Guide). When this flag is set, only one instance of the OBO agent will receive the request, and the load is spread round robin across your cluster.

Redundancy

If deploying an OBO subscription manager you will want to deploy multiple instances for redundancy, regardless of scaling concerns. There are a few ways to do that. One approach is the same approach as outlined just above in the Capacity Planning and Scaling section, namely the client applications put the DTO flag on messages and the Solace router will round robin the requests to multiple instances. If one fails, the remaining instances continue to operate.

The second approach uses guaranteed messaging, with multiple agents binding to the same queue. If you are only interested in redundancy, make the queue an exclusive queue and your backup OBO subscription manager will take over if the primary one fails. On the other hand, you can achieve both redundancy and scaling by employing a non-exclusive queue which has the same round robin effect as the DTO flag does in direct messaging.

Summary

Is your solution a good match for an OBO subscription manager?

If your solution has any of the requirements that this post has identified, consider using an OBO subscription manager in your solution. It is a winning pattern that can help you provide an elegant auditing solution, abstract physical topic names from your applications, decouple applications from third party entitlement engines and/or databases, and help you scale your applications up to meet the needs of virtually unlimited entitlement management requirements.

In a future blog post, (now published!) I will walk you through the mechanics of an OBO subscription manager and a client, complete with source code.

I welcome any comments, suggestions or ideas you may have about this. Perhaps you have a use case for an OBO subscription manager which you’d like to share with the community?