The 6 Problems with Cloud That Solace Helps Solve

I recently joined Solace from the cybersecurity industry for some pretty good reasons – chief among them is that we are about building something. Or rather helping customers to build things. Security was about protecting things, and when it worked best nobody knew it was there; but when architects and developers build things with Solace technology – everybody sees it.

A colleague suggested that I turn my thoughts into a post that could benefit others new to event-driven architecture, and here we are… my explanation of why I chose to work at Solace.

Industry at an Inflection Point

We rarely think about the functional infrastructure that is hidden in the walls of our home. We love that shiny new quartz countertop or internet-connected dishwasher – but the sizzle on the steak can’t exist without the plumbing that makes the kitchen work. Plumbing that is high quality, reliable, and installed by a professional. It makes the pretty parts work and ensures that the new kitchen doesn’t cause the upstairs shower to lose pressure.

Taking this metaphor to the application world, people can do their banking on a phone. Retail merchandisers can adjust inventory based on real time demand for it. Even an industry you would think of as low-tech, like ocean shipping, can reduce fuel consumption by adjusting boat speed in response to port backlogs. The fact that any of this even works is amazing. The fact that it works at any kind of scale, instantly, and around the globe is a testament to human ingenuity.

To the average non-technical person, this is taken for granted – and let’s be honest, it is much easier to build applications than it used to be. But as some parts of developing applications get easier, the demands that we ask of them grow as well.

Specifically, if a system was built with compromise (as most of them are) then scaling can come at a heavy cost. I chose to work at Solace because we demonstrably mitigate this with event-driven architecture better than any other option. I see this when our customers in banking, retail, technology, and other industries are evaluating and choosing Solace. They are pretty discriminating and very knowledgeable.

Where Does the Complexity Come From?

Cloud computing is well into a point of global inflection in how applications are built, deployed, and managed. And with rapid change comes opportunity. Here are a few facts:

• 92% of organizations have adopted a multi-cloud strategy
• 82% are hybrid-cloud
• 80% self-report that they have some maturity in the cloud
• Only 13% consider themselves as having full maturity with event-driven architecture

To learn more, read the results of a 2021 survey in an eBook called The Great EDA Migration.

By combining cloud computing with the new demands at the edge enabled by IoT and 5G, you create exponential growth in complexity and quantity that can be solved with a middleware event layer. And helping people to solve big complex problems is just plain interesting.

Understanding What it Means to Scale

Since my thesis statement for this blog is why I work at Solace, I’ll use a personal example about doing work at scale. In a past life, I used to sift through literally billions of individual events to find a story. We didn’t operate a full security operations center, but some of our activities looked a lot like one.

To be clear, I wasn’t a security analyst, I simply needed to sift through information to find and tell stories about cybersecurity or alert customers in response to a new global security threat. When you are talking about threat data you are talking about something harder to detect than the proverbial needle in a haystack. You don’t even know if you are looking for – a needle, a pebble, a jewel, or even a slightly discolored piece of hay.

In this instance, the data that mattered would be represented in only a few bytes of information. Yet here I was sifting using an ELK instance (Elasticsearch + Kibana) on the entire dataset. It was hard, it was slow, and it was expensive from a storage and compute perspective. You quickly learn that so-called free (or cheap) cloud services get costly as they scale to even moderate workloads. In short, the decisions we made in the original deployment solved a specific problem that customers had but didn’t plan for my use case. Since it wasn’t event driven it couldn’t meet my use case, but the job still had to get done. It just wasn’t fun, nor was it efficient.

Those who work in enterprise architecture, networking, or infrastructure understand scaling problems in a way that application development teams, users and business unit managers don’t. And until it doesn’t work right you don’t understand how much work went into making sure it did.

In short, architecting a solution that solves today’s problem while being ready for tomorrow’s unknowns requires organizations to understand how to connect applications that can scale. This is not a “nice to have” in planning your next epic.

Technology Problems in the Cloud

These are the top six problems I could come up with that involve technology and the cloud.

1. Scalability: The quick, highly visible cloud app wins don’t scale well if not planned for growth
2. Hybrid/Multi-Cloud: Certain public clouds are better suited to certain problems, so you need to get them working together.
3. Cross-Functional Integration: Organizations, especially those that serve multiple industries, need the agility to let business units to develop applications in so called, “silos”
4. Cost: Scaling in the cloud is never as cheap as promised
5. Speed: Scaling always introduces unexpected latency
6. Technical debt: As integration grows more complex it can cause technical debt that is hard and expensive for future application development.

Enter Event-Driven Architecture – and How Solace Solves the Problems in the Cloud

One of the new(er) principals of application development is that of microservices. And while it is newer, it is quite well understood so I am going to use it as an analogy. Rather than a large monolithic application, it is developed as a collection of smaller ones. This improves the agility of teams in deploying new applications anywhere. Events are similar and work alongside services to decouple the publishing of events from the subscribers of that data.

For instance, in a typical RESTful development process you may want to connect several applications or microservices to a core application. The problems can come from direct connections developed in silos. By not decoupling the event production from the event consumption in a pub/sub model you have to make several compromises.

These compromises may not be bad if you understand why you are making them. But what if you had an option to not even consider them? If you are using Solace, scaling won’t increase costs, time, and frustration across the application stack. Moreover, decoupling helps the development teams to focus on the fun work of building things and not the tedious work of addressing technical debt. Going back to liking to work here – I like that what we do helps make the work of individuals at our customers better. Because, honestly, I kind of like people.

Solace helps organizations to solve the six problems I listed above:

1. Scalability: Enables development teams to build applications independent from the source data or services so they scale on demand.
2. Hybrid/Multi-Cloud: Empowers organizations to choose different cloud vendors or architectures based on which one is best suited to the application or micro-service – this also helps eliminate vendor lock-in.
3. Cross-Functional Integration: Organizational silos can consume or share events regardless of where they are produced so they can build what their customers need.
4. Cost: Solace topics and the event mesh are both designed to scale cost effectively and demonstrably reduce total cost of ownership by reducing storage needs and application layer CPU costs.
5. Speed: Solace cloud brokers reduce latency. There is a lot in this bullet because Solace brokers can be scaled up as needed, deployed in an event mesh to bring brokers close to the consuming applications, use topic routing to optimize delivery, and more.
6. Technical debt: Solace tools reduce the cost of designing event flows and of discovering events across the organization while the topic structure provides a schema for organizing structure.

Solace has a long history with event-driven architecture where it has been a requirement in some sectors (such as financial services and telecommunications) for many years. Solace has also been ahead of the industry shift to hybrid- and multi-cloud deployment requirements by enabling organizations to deploy an event mesh of SAAS, VPC, and on-premises event brokers. And finally, by supporting a wide variety of APIs and connectors, Solace makes it easier easier to deploy multi-cloud and multi-protocol applications without vendor lock-in.

Enter the Enterprise Architect (and Their Team)

We have covered why I love working to solve the problem. We have covered why I love the Solace solution. Let’s talk about why I love making our users happy.

Big companies have enterprise architects. Small companies have development managers, or analysts, or operations managers. Regardless of how well you are staffed, somebody in the organization needs to play the role of enterprise architect – even if on a part-time or ad-hoc basis.

They need to bridge technology across complex stacks by setting the strategy and helping to implement the enabling technology. This requires making choices that have a lasting impact. It isn’t a job for a person who wants to jump from quick win to quick win or for someone who wants to “ship early and ship ugly”. It is a job for a “doer” who can think big picture AND long term about the consequences. But they are also technical people – and I really like working with them.

These folks may be (1) new to event-driven architecture, (2) doing a rip and replace of an old technology, or (3) switching to Solace because their chosen solution didn’t meet expectations at scale. Regardless of why, our customers see immediate benefit – especially in the cloud, because it is easy build and then to scale up that first quick-win. If they later want to deploy their own standalone broker instances, it is super easy to do that too.

It unleashes our capacity to do very high throughput transactions, really at the speed of light.”
CIO, Airtel

As service providers in a B2B market, we all like it when customers can sell more products or reduce costs – but these are the kinds of stories I have had to hear my entire career. Helping your customer be profitable is, quite frankly, table stakes. What drives me is that Solace-enabled event-driven architecture is foundational to organizations unlocking all kinds of different events for all kinds of previously unaddressed issues.

Companies that can trade stocks faster in a zero-sum market, companies that are using event data to reduce pollution, companies that link IoT together to optimize automotive systems and even one day reduce car accidents. These are stories that leave the world a little better than we came into it. I would like to return to my opening metaphor. Sometimes the plumber developer, working with really great plumbing products tools (Solace & event-driven architecture), can work miracles too it seems.

To conclude, my old marketing 101 professor would be upset if I didn’t add a “call to action”, so I will go with two. First, join us. We are a great bunch to work with and the growth in demand for Solace solutions requires a lot of hiring to keep up. Second, if you are looking to deploy a new application that leverages EDA then learn more about it in “The Architects Guide to Implementing EDA”.  You don’t need to be an architect to benefit from this one.