How the Hottest Hedge Funds Capitalize on Event-Driven Architecture

Architecting a highly resilient and robust data distribution architecture is of utmost importance to a hedge fund, but many firms still use a variety of messaging solutions that were not designed to meet the needs of their use cases. As a result they require external components for key features, are miserable with global message distribution, and simply aren’t resilient enough to enable a global trading system.

Over the course of my career as a data engineer at a capital management firm, a q/kdb+ developer for a major bank, and now a solutions architect at Solace, I’ve helped a variety of leading firms in capital markets implement messaging, integration and streaming solutions, including asset managers, leading liquidity providers and exchanges, the world’s biggest investment banks, and top buy-side firms including Akuna, Two Sigma, Old Mission Capital and Vanguard.

All of these firms have used event-driven architecture to build a robust, resilient, and real-time trading infrastructure. In this post I will explain the ways I’ve seen event-driven architecture help buyside firms, especially hedge funds, accelerate and modernize their systems and processes so they can thrive in their hyper-competitive market.

Architecture of a Typical Hedge Fund Trading Platform

There are multiple systems involved in a typical trading stack at a hedge fund. Let’s take an equity trading system, for example, that typically consists of market data, order and execution management system (OMS/EMS), PnL, risk, and post-trade/settlement.

Common applications that form a hedge fund’s trading system.

Each of these applications require exchanging numerous and different types of messages with each other. OMS and EMS need to know the latest prices for the securities that are about to be traded, PnL needs to know which orders have been executed to be able to calculate the PnL, Risk needs to know the prices to be able to calculate important metrics such as VaR (value-at-risk) for the positions/holdings, and post-trade needs to know the order confirmations for reconciliation and reporting.

The Importance of Infrastructure

Architecting a highly resilient and robust data distribution architecture that links all of these applications in real-time is of utmost importance to a hedge fund, but many firms still use a variety of messaging solutions that were not designed to meet the needs of their use cases. As a result they require external components for key features, are miserable with global message distribution, and simply aren’t resilient enough to enable a global trading system.

Common Hedge Fund Use Cases

At a very high level, we can classify the messages shared by applications at a hedge fund, into two popular categories: market/pricing data and trade/execution orders.

Both are quite different in nature, and there is clear contrast in three areas:

• Volume and Volatility: Market data is high volume and extremely volatile whereas trade orders data is less so.
• Tolerance for Message Loss: With Market Data, you prioritize getting latest prices at the expense of potentially losing some messages. On the other hand, there is zero tolerance for losing any order data even if it means having some delay.
• Quality of Service: In messaging terms, Market Data is delivered with at-most-once quality of service whereas order data is delivered with at-least-once quality of service.

Solace supports all kinds of use cases through direct messaging (also called best effort, in-memory, non-persistent) and guaranteed messaging (also called persistent). Direct messaging is used for high-throughput and low latency use cases such as market data distribution whereas Guaranteed messaging is for high value and lower volume trade order data.

With guaranteed messaging, high-value messages such as trade orders can be delivered from publishers to consumers with zero message loss and in-order delivery.

Key Requirements and How Solace Meets Them

Solace PubSub+ Event Broker lets you integrate all of your applications and devices across clouds, datacenters and the internet of things in an event-driven, real-time manner. Here’s how specific attributes and capabilities that make it the perfect solution for hedge funds looking to accelerate and modernize their trading platforms.

Programming Language and Protocol Support

These applications can be written in any suitable programming language—Java, C++, python, Go and JavaScript among others. They could also be dependent on different messaging standards or protocols such as JMS, AMQP, or WebSockets. Solace’s support for multiple protocols and APIs enables these applications to pick their own protocol and API and let Solace’s broker do the translation. A backend Java feed handler can publish market data using AMQP or Solace’s proprietary protocol, SMF, and an HTML5 PnL dashboard can subscribe to these prices using WebSockets over JavaScript.

Sophisticated Routing

To efficiently get low latency market data and time-sensitive transactional information everywhere it needs to be, hedge funds need a way to route information to any and all applications and users that need it, anywhere in the world. To meet this need Solace supports rich hierarchical topics with wild card filtering. For example, a market data feed handler can publish equities data with the following topic taxonomy:
marketdata/<feed>/<asset_class>/<region>/<venue>/<ticker>/<granularity>

An example topic to publish IBM’s prices from Bloomberg would be:
marketdata/bbg/eq/us/nyse/ibm/tick

Having support for hierarchical topics allows downstream consumers, such as a PnL application, to receive filtered data. An application interested in only data for US equities can simply request filtered stream by subscribing to:
marketdata/*/*/us/>

It’s important to emphasize that the filtering is done by Solace at the broker level, relieving downstream consumer of additional compute.

Caching

Many hedge fund use cases rely on  the ability to cache latest prices. When a trader UI starts up, if some of the securities are not very liquid or the markets are closed, the UI will display the last cached price. Solace PubSub+ Cache can be setup alongside Solace brokers to work seamlessly with Solace’s APIs. An application will simply connect to the broker as usual and will receive last price from the cache without any additional work.

Eliding

Another requirement that comes up in many hedge fund use cases is a kind of rate limiting called eliding. Some consumers don’t need and/or can’t handle every tick in a stream, for example a trader UI that is used by a human does not require prices to tick at the rate of hundreds per second because the human eye can only process so many updates in a second. Solace’s brokers let you set the rate at which each consumer receives messages, say every 10^th or 100^th update.

Timeseries Databases

Eventually all this market data and trader order data needs to be collected in a timeseries database, such as kdb+ or OneTick, for real-time and historical analysis. Solace offers plugins for kdb+ and OneTick  that make it easy to ingest timeseries data for further analytics.

Globally Distributed Operations

Many of the large financial firms have a global presence, especially in major trading hubs such as New York, London, Singapore, Hong Kong, and Tokyo. A typical deployment involves sharing messages across all these hubs with minimal latency and high resiliency.

One of the world’s biggest hedge funds uses Solace to distribute trader order updates published by local instances of their OMS over Solace’s global event mesh. The order updates are consolidated back to their primary datacenter for Risk analysis of their positions in real-time.

HA pairs deployed by a hedge fund in 3 trading hubs globally to distribute trade orders.

Such a deployment requires an event mesh with Solace message brokers deployed in each trading hub and connected to each other. Local applications connect to their local broker to publish and consume messages.

Each of these nodes are typically deployed with high availability (HA) and, if required, disaster recovery (DR) in mind. Solace brokers can be deployed as HA pairs to provide resiliency within a datacenter and as DR mates to provides resiliency across datacenters. For example, a firm might have 2 brokers in a New Jersey datacenter for HA and another pair of brokers in Virginia for DR. If there is a broker failure in a datacenter, the backup broker will take over within 30 seconds and if both brokers are out-of-service (due to a datacenter outage), then applications will failover to the DR datacenter within minutes (though typically DR requires a lot of coordination with other applications and usually takes longer).

Deployment Options and Form Factors

Solace offers its brokers in three form factors that meet any need and can be deployed in any on-prem, co-lo or cloud environment: hardware, software, and as-a-service.

Hardware

How you deploy your brokers depends on your exact use case, but at a hedge fund where brokers are expected to support throughput of millions of messages per second and fan-out to several downstream consumers, a turn-key hardware solution is the most ideal.Despite software’s rise in popularity, Solace brokers are still deployed in their hardware form factor with custom designed FPGA blades to support high throughput and low latency.

Software

But that doesn’t mean every use case at a hedge fund requires Solace’s hardware appliances and that’s why there is an option to deploy the brokers in their software form factor as docker containers. Of course, their performance is different from the hardware, but they are easier to deploy and get started.

Sample hybrid-cloud deployment at a hedge fund for leveraging AWS for risk calculations.

Cloud

Even if your applications are not currently in cloud, you most likely have a mandate to migrate your applications to the cloud in near future. As applications move to the cloud, the messages that power these applications need to follow. Messages need to traverse on-prem infrastructure and be delivered to the cloud to be consumed by cloud applications. Solace’s software brokers can be easily deployed in any of the cloud providers (either as self-managed or as managed service via Solace Cloud) and linked to on-prem brokers (regardless of their form factor) to form an Event Mesh. Your application code doesn’t need to change at all. Simply point your cloud applications to cloud broker and it will be able to publish and consume messages from the cloud broker.

Conclusion

Architecting a highly resilient and robust data distribution architecture is of utmost importance to a hedge fund, but many firms still use a variety of messaging solutions that were not designed to meet the needs of their use cases. As a result they require external components for key features, are miserable with global message distribution, and simply aren’t resilient enough to enable a global trading system.

Save yourself the headache of managing a messaging infrastructure that keeps disappointing you and causing your firm monetary loses due to downtimes by switching over to Solace PubSub+ Event Broker. Lots of leading hedge funds are using Solace brokers, so why aren’t you?

What’s Next?

If you would like to learn more about our offerings and how world’s leading financial institutions leverage Solace for pertinent hedge fund use cases, book a consultation and we would be happy to set up a quick call.