Investment Banking and the Middle Office Shock Absorber

Equities trading is the poster child for applications that demand the distribution of lots of data with extremely high performance – latency in the low microseconds, reaching for nanoseconds, and with as little “jitter” as possible even through unpredictable bursts of trading activity. But when it’s time to send order and trade data to the middle office systems responsible for non-real-time functions such as risk management, OATS reporting, etc., the prime directive shifts from speed to certainty. Every single message must be delivered to lots of recipients, many of them slow consumers, without putting “back pressure” on high-speed front office trading applications. This is why equities trading and OMS systems need a shock absorber.

Ever since algorithmic/electronic trading revolutionized capital markets, the “need for speed” has driven the development of networking technologies like cut-through Ethernet switches and 10 GE NICs that support kernel bypass, and encouraged techniques like running multiple apps on multicore servers with overclocked cores. The demand for peak performance has also spurred debate about the use of FPGAs vs. software for algos and feed handlers, and—near and dear to our heart—the benefits of peer-to-peer vs. hardware broker based messaging.

Note that all these developments and debates are in the space of the well-documented front office – where orders are received (or generated if you’re on the buy side of the equation) and trades are completed. But what about the middle office? In every trading architecture there is a need to move all trades and often order events from the front office to “outside” the front office. This first hop out of the front office is what many call the middle office, the home of functions such as trade data capture, near-real-time risk management, OATS reporting, order/trade replay functions and sometimes order workflow monitoring.

For middle office applications, microsecond latency and being able to deal with instantaneous bursts of events is not necessary.  Take OATS reporting, for example – you need to process and report the daily volume of orders and trades by the OATS reporting deadline, but that gives you until 5:00 am the next day.  Other middle office functions such as risk monitoring demand much higher “near real time” performance, because knowing about some action you should have taken after markets close is pointless, but you’re still not talking about the need for microsecond latency.

So you’ve got a massive volume of real-time events occurring at huge and unpredictable rates in the front office driving a flow of information to many middle office applications with very different levels of performance. And all these events must be delivered to the target application, without loss and at a rate each can deal with, even if you’re utilizing peer-to-peer architecture in the front office. You need a shock absorber to  absorb this huge volume of information, persist it so you never lose a message, fan it out to many downstream applications at a rate they can manage, even slow consumers, all without putting back-pressure on those fast front office applications.

As mission-critical as the need for a middleware shock absorber is, it’s just one of the requirements for middle office messaging in trading/OMS platforms.