One of the hottest topics in the capital markets these days is the migration of market data distribution to the cloud. It’s appealing because doing so can improve your ability to scale up and down as volumes fluctuate, reduce costs, and monetize proprietary data about trading activities. Many financial institutions, however, are struggling to get started for a variety of business and technical reasons.
That’s why I recently teamed up with my colleague David Eden (who has over 30 years experience selling market data technology) to write a new white paper in which we break down the business case for moving market data to the cloud, explain the requirements and roadblocks associated with doing so, and share some idea for how to meet and overcome them.
In the 16-page whitepaper, Accessing Real-Time Market Data in the Cloud with Solace: The Business Case for Cloud-Delivered Market Data, we first explain why investment banks want to migrate their market data distribution to the cloud, and describe the challenges they face in doing so.
Why Migrate Market Data Distribution to the Cloud?
There are many commercial and technical drivers that are leading businesses towards cloud-delivered market data. A sample of some key drivers from the white paper include:
- Faster time-to-market for market data products and faster deployment than on-prem datacenters.
- Fostering an environment of innovation and experimentation.
- The ability to focus on product delivery and not burn resources acquiring and maintaining assets in data centers.
- Reduced capital expenses and servicing smaller clients more effectively and efficiently, with less infrastructure.
- Increased efficiency and lower latency offerings from cloud providers.
- Reduced dependency on large market data infrastructure providers.
Overall, market data distribution technology requirements are very well served by existing on-prem deployments and will continue to be so for years to come, especially after decades of investment in robustness, resilience, latency, and volume handling. However, the landscape is changing, and cloud solutions are being adopted for specific use cases. Aspirational initiatives that were previously uneconomic have now become viable through cloud computing.
Challenges with Cloud-Delivered Market Data
From potential latency to performance issues to the steep learning curve for developers and architects, there are a number of challenges associated with migrating to the cloud. However, the biggest challenges we have seen in our research are cost-related. Any wholesale migration to the cloud is not going to save money in the short term due to the cost of dual running both cloud and on-prem environments and running a multi-cloud environment can also be discouraging due to significant costs.
Over time, the benefits of a flexible environment, reduced support cost, and deprecation of on-prem should be considerable. The lost opportunity cost of not being in cloud and attracting or retaining customers should be considered when deciding on your path forward. We cover more details around these challenges in the white paper.
Considerations and Requirements for Market Data Distribution in the Cloud
The meat of our white paper addresses a specific example where we define a “simple market data service”, set requirements, and evaluate whether a cloud-native service would suffice. The requirements we set for a service with moderately high-volume market data lead us to the four main issues that cloud-native services have trouble overcoming:
- Latency
- Quality of service (delivery)
- Message order
- Multi-cloud capability
After reviewing the example requirements for latency, volume, recipients, filters/subscriptions, delivery needs, message size, and more, we dive into the limits or challenges you might run into by using a cloud-native service for market data distribution, something they arguable were not designed to handle.
We wrap things up where you might expect: explaining how you can use an event mesh powered by Solace PubSub+ Platform to deploy a real-time market data service in the cloud. We use the same example requirements for the simple market data service and demonstrate how they can be met by using event brokers to bring tried and tested ‘on premises’ style messaging into the cloud. An event mesh, or network of event brokers, can be used to move data between cloud and on-premises deployments, or vice-versa. We provide an example of how to scale horizontally for global distribution and 1M messages per second.
Accessing Real-Time Market Data in the Cloud with SolaceDiscover the business case and practical how-to tips for deploying market data in the cloud with an event mesh for exchanges, trading systems, data providers and banks.
We put a lot of time and energy into it, so I hope you find it informative!
If you have any questions, send me a message or ask away in our developer community forum.
Explore other posts from categories: For Architects | Use Cases
Subscribe to Our Blog
Get the latest trends, solutions, and insights into the event-driven future every week.