Ankil Patel (Guest Author)
Originally published on the Macquarie Engineering blog at medium.com by Ankil Patel, Engineering Director at Macquarie Group.
A broken pipe on a holiday weekend is every homeowner’s worst nightmare. The gush of water, the desperate call to a plumber, imagining the unseen horrors beneath the floor, and hopefully relief when it’s over and wasn’t as bad as it seemed. And one potentially painful final step: paying the plumber with your mobile app and seeing a reassuring “transaction successful” message on the screen. Macquarie’s event platform ensures that the payment goes through quickly so you can move on with your weekend plans. And it’s doing the job even better since our migration to the cloud.
Macquarie’s event-driven architecture distributes real-time, actionable information around the globe, enabling better decision making, meeting regulatory requirements and improving our customers’ experience. When a Banking and Financial Services (BFS) customer initiates an interbank transfer, to pay a plumber for example, the event platform routes the information instantaneously to meet the requirements of the New Payment Platform. When a Commodities and Global Markets (CGM) trader buys a stock in London, the event platform routes that event to dozens of middle office applications and eventually to data storage in Sydney, guaranteeing that information won’t be lost along the way. The event platform integrates hundreds of geographically distributed applications through loosely coupled asynchronous data movement.
Event brokers from Solace have been the foundation of Macquarie’s event platform since its inception in 2013. The first iteration of the platform used hardware appliances: single-purpose physical hardware, finely tuned to process events at high speed. This solution performed exceedingly well. But with Macquarie’s goal of moving 100% of its infrastructure to the cloud by end of 2022, it became apparent that the future of the event platform was not hardware appliances sitting in on-premises data centers, but rather in software event brokers residing in the cloud.
Migrating the event platform was a complex project with a tight deadline. But the result was worth it: Macquarie became the first financial institution to establish a Solace-based event platform running exclusively in public cloud! Since the migration, the platform has moved more than 8 billion events per month.
Moving the event platform to the cloud had multiple benefits:
- Rapid deployment of new brokers and updates
- Faster, cheaper, more secure event movement
- Intelligent scaling that maximizes value
Planning the Project
The cloud migration project delivered benefits across the enterprise. The event platform integrates hundreds of enterprise applications running on-premises and in the cloud, supporting operations in more than twenty-five countries across the world. The critical nature of the events means that Macquarie’s businesses are dependent on the event platform 24×7.
As a result, the migration could not incur unplanned downtime and needed to minimize planned outages. Adding to the importance: The Solace appliances were reaching end-of-life, which gave our small migration team a time frame of just eleven months. Planning started in earnest.
The first decision was selecting the cloud-based event brokers. Although there were numerous competing event brokers, Solace was the natural choice given the success we’d had with the appliances. Fortunately, Solace event brokers exist in a variety of form factors in addition to the hardware appliance, including a Docker container. The container seemed to be a perfect fit, given the eventual destination of Macquarie’s Amazon Web Services (AWS) virtual private cloud (VPC). However, in contrast to cloud-vendor specific solutions (like Amazon’s SQS), Solace can be deployed on multiple clouds, giving us the ability to pursue multi-cloud solutions in the future.
Our deployment topology of Solace PubSub+ in AWS
Rapid Deployment of New Brokers and Updates
Moving from the hardware appliance to a Docker container also meant we could start taking an infrastructure-as-code (IaC) approach. IaC removes manual, error prone deployment steps. We built a completely automated IaC pipeline that would deploy a cluster of highly available brokers in AWS using Macquarie’s cloud deployment platform (Arturo). These pipeline configurations (such as CloudFormation templates for containers deployed to AWS), dictate the broker and cloud configuration, making sure that it is consistent for every deployment.
Rather than taking a big-bang approach of moving all of Macquarie’s brokers to the cloud in one shot, the migration took place over course of nine months, one business unit at a time. For each migration, we used Arturo to deploy the new cloud brokers in AWS. Then at the given time, applications sending and receiving events from the brokers switched from on-premises broker to cloud brokers, no code changes necessary.
Post-migration, the investment in infrastructure as code continues to pay off. Like all software, Solace broker receives regular upgrades and patches. With IaC, those updates are handled through Arturo, including the ability to update brokers one at a time, to keep traffic flowing through the platform. While the Administrative consoles on the broker can still be used for troubleshooting and monitoring, no configuration changes are allowed, which enhances security, stability and accountability. IaC enforces the integrity of the environment as an additional benefit of running the brokers as immutable infrastructure with read-only console access for monitoring & troubleshooting only.
As our team completed the nine-month migration, additional benefits came into view. With the Macquarie-wide cloud migration in full swing, numerous applications that send and receive events now sat in the same cloud. The change in physical location benefits us in numerous ways:
- With less network distance to span, events move even faster between applications, spending less time on the wire, pushing even more responsiveness
- Events now don’t need to enter and leave the cloud as frequently to reach their destination. That reduces ingress and egress charges, a significant cost associated with cloud computing
- There is less interaction with applications outside our virtual private cloud, which means less complex security configuration.
Fine-grained scaling for maximum value
The move from appliances to software brokers also increased the platform’s flexibility and agility. While the physical event broker appliances were powerful and capable, for cost-efficiency several lines of business shared a single broker. Most of the time this worked fine, but it occasionally introduced complexity. For example, commodities markets and stock exchanges are closed on weekends, which makes it an excellent time to perform routine maintenance. Meanwhile, banking customers need rapid access to their money on the weekends (for emergency plumbing repairs, for example). So planned outages on weekends is a no-go.
Moving to software-based brokers means that each line-of-business has their own dedicated broker resources. As a result, maintenance windows can be customized to their individual needs — BFS brokers are patched in the middle of the night, while CGM’s window is on weekends. But because the individual brokers are bound together into an event mesh, the global connectedness remains.
As the platform evolves, this flexible topology will be key. Surges in event traffic or additional integration requirements can be addressed by adding additional brokers to the line of business (horizontal scaling) or by additional CPU and RAM to an existing broker (vertical scaling).
Conclusion
If data is the lifeblood of Macquarie’s business, then our event platform is its beating heart — enabling the movement of that lifeblood among hundreds of applications in real-time. Migrating the event platform to the cloud reduced the costs associated with the platform, while simultaneously making it more agile, resilient and responsive.
But there is still more to do. Analysts from Gartner hail event-driven architecture as a key foundation for managing the deluge of real-time data from sources like the Internet of Things and mobile devices. To meet those expectations our event platform must continue to mature. In the coming years we expect to:
- Expand our self-service options, allowing developers within Macquarie to create their own event broker objects efficiently and securely
- Link the cloud-based Solace event platform with existing IBM MQ messaging and Kafka installations, to create an even more comprehensive event mesh that can address even more use cases.
- Leverage new architectural tools that can visualize, manage, design and govern events at an enterprise level
- Explore moving to a messaging as a service model
With a firm cloud event platform, Macquarie is well positioned to push our innovation even further.
Subscribe to Our Blog
Get the latest trends, solutions, and insights into the event-driven future every week.