Trading Firm Creates a Hybrid Event Mesh to Benefit from AI Cloud Services

It’s no secret that product marketers love to tell stories about customers using our products, but this story has me more excited than usual.

Tan T-Kiang (TK), Grasshopper’s Chief Investment Officer, seemed amused when I asked him several weeks ago:

What business impact does a Solace-powered event-driven architecture have on your trading firm?”

How Tan T-Kiang looks when I’m not asking him questions.

Why was he so amused?

Because trading firms have been using event-driven architectures for decades! TK explained that investment banks couldn’t predict what applications they would need to share information with in the future. Ten years ago, he didn’t know that Grasshopper would want to use GCP’s BigQuery and Cloud Dataflow to make near real-time trading decisions and develop trading strategies.

But what he did know years ago is that he needed to build a trading platform that enabled different applications to share events across different environments; he needed the platform to be flexible and dynamic so that it could take on and adapt to new technologies as they evolved. So that’s what TK and his team did.

It paid off. In 2011, Grasshopper built an event distribution platform to exchange real-time market data and transactions across many exchanges in Singapore and Tokyo, and move data from a variety of applications and algorithms across asset classes.

TK and his team selected Solace’s messaging appliances because he had used them successfully in previous companies. He was introduced to Solace in 2007, and he thought our integration of middleware into a device was innovative. To TK, our PubSub+ appliances provided seamless data movement using open standards and protocols and was capable of sharing events in real-time with applications that he hadn’t thought of yet.

Fast forward a few years and technology advanced dramatically.

Grasshopper wanted to move data between their on-premises locations and Ahab, their new cloud-native application. Ahab, an Apache Beam-based Java application is a high-performance data processing pipeline that uses real-time market data to calculate the value of the order book—a dynamic list of buy and sell orders that dictates stock pricing—from real-time market data received from multiple stock exchanges. As the order book is the foundation for many trading and analytics applications, high performance was truly mission critical.

From there, Ahab ingests the result to BigQuery, GCP’s fully managed data processing and analysis tool, utilizing the tool’s availability and scalability. Though they had to make many other changes, the one thing they didn’t need to change was that their trading platform was built on Solace technology. (Let me be clear: I’m not saying Solace stood still. In addition to the appliances, enterprises can also take advantage of software brokers or as a service.) Grasshopper ultimately used PubSub+ Event Brokers to create an event mesh – a modern messaging layer that can be deployed across every environment and component of their company, to stream events across them all.

Here’s TK talking about his use of PubSub+ for scaling event-driven machine learning capabilities at-scale:

How it works

Let’s drill down into the architecture at Grasshopper.

The Solace PubSub+ Event Brokers (in GCP and on-premises) create a hybrid cloud event mesh allowing orders published to the on-premises brokers to flow easily and dynamically to the cloud PubSub+ brokers and SolaceIO connectors. The SolaceIO connector (running in Google Cloud Dataflow) consumes this market data stream and makes it available for further processing with Apache Beam running in Cloud Dataflow. The processed data can be ingested into Cloud Bigtable, BigQuery and other destinations, processing in a pub/sub manner.

“Why PubSub+?” you may ask. “Couldn’t Grasshopper use some other technology?” In TK’s own words:

Solace provides the reliable, robust event distribution we need to exchange all kinds of real-time market data and transactions across many exchanges. Our decision to use PubSub+ to incorporate cloud-based machine learning into our system was easy given Solace’s hybrid cloud capabilities.”

Grasshopper has experienced many benefits already, including:

• Improved speed and sophistication of their trading;
• Increased on-demand scalability while retaining latency, performance and connectivity requirements; and,
• Cost-effectively complementing the expertise of their algo/quant developer.

Solace PubSub+ can help you create a robust and dynamic event mesh as you shift some applications or workloads to the cloud. If you’d like to learn more about Solace PubSub+, PubSub+ Cloud, or a Solace-enabled event mesh, contact us.

Get the Grasshopper Success Story PDF