Solace Systems Message Routers have been deployed by innovative companies of all sizes as the messaging foundation for their enterprise services bus. Our technology ties together applications across datacenters via event driven data flows, so their most mission-critical high volume data is already flowing through Solace Message Routers. As a result, when these organizations embrace new technologies they call upon us to help integrate them into their ESB.  It is as part of these efforts that we have been working for some time with Cloudera as an enterprise grade big data solution. Using Solace to feed Cloudera data workflows has given joint customers superior insight into any data, from any app, in real-time, with an enterprise mindset.

cloudera-certified-logoBased on these successful deployments we decided to formalize our integration with Cloudera to make the integration easily repeatable for our joint customers, and we are thrilled that we recently achieved certification with Cloudera’s Enterprise Hadoop solution (CDH).

The goal of this partnership is to let Solace customers seamlessly integrate Cloudera’s big data solutions into their enterprise message bus. Doing so lets companies analyze enterprise data in motion in near real-time for fast time to insight without using dedicated messaging infrastructure to place data from across the enterprise into an HDFS data lake or requiring bespoke integration to CDH.

This integration provides a robust enterprise grade data flow solution to easily ingest data from a variety of environments without having to build and manage a variety of messaging and log transfer solutions.

We have seen two approaches for adding big data into enterprise data architecture: overlay and integrate. To learn more about the pitfalls we are seeing as people try to overlay big data on their enterprise bus, read this piece about the messaging proliferation problem. The overlay approach is usually taken because the existing messaging infrastructure is not up to the task of taking on big data sources and infrastructures. Solace has proven to be capable of integrating big data into existing messaging architectures without the need to overlay, which means your data flows into Cloudera will inherently have the advantages of high throughput, low latency, high availability with fault tolerance and disaster recovery.

Cloudera-Solace-DiagramTogether Solace and Cloudera provide an excellent solution for integrating big data into your enterprise service bus.  Solace provides real-time messaging while being both highly available and highly persistent, while Cloudera provides big data infrastructure for near real-time analytics and processes for deep analysis and visualization of data. Together we provide:

  • Complete set of integration points from all data sources, mobile, web, IOT, app servers, mainframes, and data stores into a bus to feed the big data solution.
  • Integrated queuing points to provide real-time analytics and archiving into HDFS for further analysis.
  • Federated security to enable veracity of data from source through to and including landing in HDFS.
  • Access Controls to restrict which application is able to produce what data, as well as where data lands within HDFS and who is able to run analytics on the data.

What’s next?

Solace is continuing to work with Cloudera and our existing customers to develop optimized manageability solutions that enable the seamless deployment and monitoring of Solace/Cloudera integration points.

Learn More:

Ken Barr

Ken Barr is a Senior Product Integration Architect working with the Solace CTO group. He's focused on exploring areas in which our customers would benefit from Solace innovation, then defining how these new technologies fit into Solace’s product lines.

Prior to joining Solace, Ken was a Technical Lead at Cisco Systems holding several roles in carrier core routing business units including QA team lead for initial implementations of IPv6 and bringing next generation IOS called IOS-XR to the core routing platforms. Preceding Cisco, Ken was a Communications Electronics Engineering Officer for the Royal Canadian Air Force responsible for operational management of the National Defence Headquarters Metropolitan Area Network.