The Advantages of Using Solace for an Event-Driven Data Mesh

I don’t always use event-driven data mesh. But when I do, I use Solace. I might not be “the most interesting architect in the world,” but data mesh has certainly emerged as one of the most interesting trends in information technology today, promising an escape from globs of unusable information sitting at the bottom of a data lake. But making the leap from concept to implementation requires a solid underlying infrastructure.

As defined by its creator Zhamak Deghani, data mesh “is a sociotechnical approach to share, access and manage analytical data in complex and large-scale environments – within or across organizations.” Data mesh shifts the focal point for data analytics from a centralized team in charge of an enterprise data lake to “data products”. Data products are created and managed by the domain teams that know their content best.

It all sounds great, but one of the things that makes data mesh so interesting is that it doesn’t prescribe an implementation. In fact, data mesh is aggressive in not prescribing implementations:  “Data Mesh must embrace the diversity of the use cases and their unique modes of access to data.” Where does that leave you, the person tasked with making it real?

When Event-Driven Data Mesh is the Right Choice

Listen, I agree: every organization is different, and should use the tools, infrastructure and organization that best suits its data and people. But I will also say this: event-driven architecture deserves serious consideration as part of your data mesh mix.

To understand why, here’s the Deghani’s description of data mesh’s reason for being: “More than ever now, organizations need to have the pulse of their data and the ability to act quickly and respond to change with agility. What does this mean for the approach to data management? It requires access to the quality and trustworthy facts of the business at the time they happen. The data platforms must close the distance—time and space—between when an event happens, and when it gets consumed and processed for analysis.”

Making the world smaller and analysis faster is a hallmark of event-driven architecture and event-driven data mesh in particular. Following that reasoning, use cases like these benefit the most from an event-driven data mesh:

• Analytical data products change rapidly
• Decisions based on those data products also change rapidly
• Domain teams have some expertise with asynchronous information

Why Solace is the Right Choice for Enabling Event-Driven Data Mesh

If the situation calls for an event-driven data mesh, the underlying connectivity provided by the event broker becomes essential. Solace provides the capabilities you need to implement a rock-solid event-driven data mesh. Here are few key reasons why.

Micro-target domain data product events

An event-driven data mesh constantly propagates huge amounts of information around your enterprise. In the firehose of data, much of it may not be relevant to consumers of your data product. To prevent overload, the event broker infrastructure must offer multiple ways of narrowing in on only data that a particular consumer is interested in. For instance, what if a domain only cares about a certain geographic location, or only care about online sales–or perhaps only online sales from a particular region?

Solace can fine-grain filter events, before they reach consumers, to reduce the load. Only want European online orders?  Done.

Like how consumers can (and should) be picky about their events, what consuming domains are interested may change over time. Shifting business requirements and team responsibilities should dynamically feed into self-service data product changes. Suddenly in charge of formulating online marketing for Egypt? With Solace, adding the Middle East to the online order data flow only requires the consumer to update an event subscription. No longer interested in the data? Simply have the application stop requesting the information and it stops flowing.

Transparent data product access around the globe

Valuable event-driven data products can be physically located on the other side of the world. Sales projections in South America are dependent on production in Asia, which can change constantly. With Solace, multiple brokers form a globe spanning “event mesh.” Event-driven data products can be in the cloud or on-premises, it doesn’t matter. And the same intelligent routing and filtering applies no matter the origin: only essential data flows to each consumer, saving time, money and processing power.

Flexible Protocols

Frequently event-driven architectures aren’t considered because domain teams don’t have the tools or use incompatible protocols. In those cases, REST becomes a default choice, even though it may not the be the optimal solution. Solace allows access to data products through a diverse collection standard protocols like AMQP, JMS and MQTT. It even can convert events into trusty old REST for domain teams that need it.

Rock solid reliability and security

Given the importance of analytics to decision making, the infrastructure needs to ensure high availability, guaranteed message delivery, fault tolerance, and disaster recovery. Solace provides a central control panel that ensures the data mesh is up and running and crucial data is not lost.

Design-Time/Governance capabilities

Finally, the move away from centralized control of data puts a huge emphasis on collaborative governance. If there’s no “data guys” sequestered away making decisions for everyone, representatives from each domain need to work together to design, maintain and govern the data mesh. Let’s hear it from Deghani:

“Even though data products are the responsibility of each domain team, there are still common concerns that need standardized solutions. Things like security and privacy enforcement, meta-data management and more will need to be determined at the organization level. The key is that representatives from the domain teams should play an integral role in the decisions, not defer them to “experts” in a separate group.”

That type of collaborative governance can flourish with the right tooling. Solace’s Event Portal provides a central location for collaboration on an event-driven data mesh. When existing data products change, the complex impact on the data mesh needs to be clear. And to justify the added effort of new data products, they need to be visible to the rest of the enterprise. The Event Portal provides a graphical, searchable, collaborative working environment to do just that.

Getting Down to Business

As the business environment grows even more complex, feeding quality, timely data into analytics becomes even more crucial. But existing data architectures like data lakes have proven inflexible in the face of these increasing demands. A rethink was necessary. The result was data mesh, the right architecture for analytical data. When the situation calls for an event-driven data mesh, Solace provides a solid infrastructure for your precious data.