Could the “Messaging Proliferation” Problem Happen to You?

If you work in capital markets, you know that most buy side and sell side firms operate lots of different messaging products across their business. It’s not unusual for a firm to have six to ten different messaging technologies in their production environment, from different vendors, as a legacy of M&A and developed internally. As a rational technologist, I can hear your internal line of questioning: How does this happen? What drives this type of fragmentation? Is it as bad as it sounds? And you’re probably thinking this could never happen to you.

But if you’re implementing big data or cloud, it probably already is. I will explain why, but first – how did capital markets get to this point and what are they doing about it?

Not so long ago, financial trading was done by customers talking to their brokers who talked to traders who placed an order to make a trade. Customers would talk to their brokers about stocks, ask about the current trading price, and then decide if they wanted to buy or sell some quantity of shares. A single technology – the telephone – was used to convey all pricing data, order request and trade execution from customer to broker to trader.

Then came the digitization of trading, which saw the telephone replaced by specialized middleware for each function:

1. Real-time distribution of prices for equities and other asset classes required high rate fanout to many consuming applications, so multicast-based messaging like TIBCO RV came along to let people see stock prices change in real-time. Of course better, faster price distribution technologies came along over time.
2. At about the same time, orders and trade completion were made electronic. This required persistent messaging because it is unacceptable to lose an order or trade. This requirement gave rise to broker-based messaging technologies because distributed multicast solutions couldn’t meet the needs of managed, stored, high-value order and trade data. JMS messaging solutions proliferated to fill this need, but sometimes they weren’t fast enough so buy side and sell side firms built their own messaging for this function.
3. All these trades need to be settled to counterparties and to the bank or broker’s customers, reported to regulators, etc. These communications are often handled by legacy or mainframe systems that are not at all real-time, and for many banks they were often connected using Websphere MQ.
4. Banks then created Single Dealer Platforms to offer more value to their corporate and high net worth clients and make their offering more “sticky”. Many also embraced the use of web technologies such as HTML5 for their internal trader desktops and for portfolio monitoring, replacing thick client applications. These required the deployment of web streaming technology.

In reality, most large capital markets participants have a lot more than four messaging technologies because different lines of business made their own decisions, M&As brought together multiple systems that used different technologies, and some early technologies were superseded by newer, better options but not all of the older deployments were replaced.

Duplication and fragmentation always creates waste. In this case, the existence of multiple messaging technologies costs the business by requiring different processes, procedures and tooling to run them in production, demanding less transferable application development skills, needing to bridge messages between the technologies, networking requirements, governance for high availability, disaster recovery, security, etc.

In the early days of electronic trading, having different middleware for different functions was necessary due to the different requirements of the different flow, the fast pace of the sector and the race to zero latency.

A key Solace value prop is our ability to unify all these different types of message flows onto a single messaging technology. We also included high availability, replication for disaster recovery, WAN optimization, security and deep management visibility so that our clients didn’t need other technologies around the messaging system to operate it in production.

It’s not unusual for us to consolidate 3-4 different technologies with a single Solace infrastructure – which in turn results in simplification, application agility as well as increased platform robustness and performance.

This is where your inner voice is saying –but I’m not in capital markets, and I don’t have nearly that duplication and fragmentation problem with my messaging technology.

As I hinted before, big data and Cloud are two horizontal technology trends that are reminiscent of the early electronic trading days in capital markets. Both are trends where specialized middleware is being proposed to solve new problems where the incumbent technology is not up to the task. The problem is that this new specialized middleware, as in the case of capital markets, is targeted at just this one new problem, so it can’t replace your current middleware and it won’t meet other new requirements in the future.

How can you avoid the impending middleware proliferation? That’s the topic of the next post in this series…