We often get asked that question. The two products have very different architectures and solve different problems. This is the conclusion many of our clients have come to as well after using both technologies. This webpage explains why in great detail. If you’re only interested in the highlights, download a summary of the differences or see the feature comparison matrix.
The Solace architecture is that of an intelligent broker with sophisticated features for event routing and streaming such as topic and wildcard subscriptions, built-in dynamic routing and WAN optimization, message priority, sophisticated disaster recovery, dead message queue, multiple open protocols, exchange patterns and QoS as well Replay – all integrated directly into the broker. PubSub+ Event Brokers were built from the ground up to help enterprises stream real-time events where they need to go reliably, securely, quickly and guaranteed – either locally and/or across hybrid-cloud/multi-cloud deployments. We do this by creating an event mesh – an architectural layer that allows events from one application to be dynamically routed and received by any other application, no matter where these applications are deployed (no cloud, private cloud, public cloud) and by providing a complete event management platform, far more than a broker, to support enterprise deployment of our event mesh.
Kafka is an open source distributed commit log that also happens to have a publish/subscribe interface. The Kafka broker is a great fit for long-term storage of immutable logs in your data layer on which you can perform analytics or data transformations (using KStream). In this regard, Kafka is more like a new type of non-relational database for your log/event data at rest. The Kafka architecture is a simple broker, with heavy APIs that only speak the Kafka protocol. Kafka does not support broker features like topic wildcards, dynamic inter-cluster routing, multiple protocols, sophisticated disaster recovery, and non-persistent QoS.
As a distributed log store, by definition it cannot support features such as priority, dead message queues, queuing or request/reply and is therefore not a generic event/data movement technology. Kafka stores logs in its immutable database that scales horizontally with more brokers and that requires adapters for most functions. As such, we often see Kafka (a data-at-rest layer) deployed as an event source or sink to a Solace-enabled event mesh (data-in-motion layer).
Background and Architectural Differences
Apache Kafka was built to solve a log ingestion problem. There was a need to consume a large amount of data from many sources quickly without impeding the data sources, then store that data and deliver it to a few large consumers, namely things like HDFS writers for a big data solution. There was a need for horizontally scaled, efficient data consumption and delivery of data at a macro level, but little requirements for fine-grained filtering or high fan-out. Apache Kafka was architected to be a very simple efficient data store and replay broker, with complexities pushed out to the more complex application APIS.
PubSub+ was built to solve demanding and diverse application-to-application communication problems at scale and with high performance. Whether it’s financial trading platforms, data distribution over the WAN for hybrid cloud, massive scale IoT connectivity, or event-driven microservices – PubSub+ was designed to solve these challenging use cases. Solving such problems requires high performance in terms of both throughput and latency, along with support for consumers that are heterogeneous in both their design and data interests. To address these needs PubSub+ offers fine grained filtering and support for many open APIs and protocols, programing languages and LAN/WAN topologies. And to facilitate the monitoring and management of these complex environments, Solace technology offers end-to-end visibility including client state, automatic protocol translation and WAN optimization.
Having looked at the backgrounds and architectural differences, let’s look at how these differences affect the application of each technology to some common use cases, namely hybrid/multi-cloud, digital transformation/bi-modal-IT, event-driven microservices and the Internet of Things.