Solace Topics vs. Kafka Topics: What’s the Difference?

May 31, 2019 Aaron Lee

Ever wondered what the difference is between a Solace topic and an Apache Kafka topic? The short answer is… quite a lot. For an in-depth comparison, watch my full video below. Or keep reading for a few key takeaways.

Before we get into the details, it helps to understand exactly what a “topic” is. Think of a topic as a concept of the publish-subscribe (pub/sub) architectural pattern. They are used to route data or events (in the form of messages) between distributed applications, often using a message broker or an event broker.

Typically, consuming applications register a topic subscription of interest with the broker. Whenever a publisher or producer sends a message on that topic, the broker forwards the message to any consumers that have a matching topic subscription. This promotes a loosely-coupled architecture, and provides a scalable, one-to-many style of communication.

Topic Implementation

Because topics are just concepts, their implementation and routing behavior is decided by specific brokers. Now, from an API perspective, Kafka and Solace APIs look pretty similar at first glance—both can subscribe and publish to a topic. However, one big difference is that consumers in Kafka need to poll to receive more messages, and in Solace consumers can follow a more event-driven design where messages are pushed to the application via a callback function.

A slide showing the differences between Kafka and Solace APIs

Kafka topics are implemented as log files, and because of this file-based approach, topics in Kafka are a very “broker-centric” concept. Messages on Kafka topics are shared across partitions, and this can result in out-of-order messages across the whole topic, and can put a limit on the number of topics you can maintain in a Kafka broker.

With Solace, topics aren’t actually configured on the broker: they’re instead defined by the publishing application at publish time. They can be thought of as a property of a message, or metadata, and not like “writing to a file” or “sending to a destination.” Each and every message could be published with a unique topic.

Looking for some examples? At the 4:35 mark below I share few real-world use cases, one with a foreign exchange trading platform and another with a connected buses platform.

A slide showing examples of topics in Solace

Managing Topics on the Consumer Side

On the consumer side, when an application connects to Solace and specifies a topic subscription, it’s done in a very dynamic, real-time way. Solace forwards the message onto the clients on a message-by-message basis as they arrive at the broker, using essentially a subscription matching engine. Because of this, there’s a strong order guarantee on topics in Solace; if two clients/consumers are subscribed to the same topics, they will always receive messages in the exact same order.

But since published topics are so dynamic, how do we manage this from a subscription perspective? You can’t subscribe to an infinite number of topics! The answer: subscription wildcards.

Solace supports both multi-level > and single-level * prefix wildcards. Take a look at some examples at 7:20:

Examples of Solace Wildcard Descriptions

Solace’s hierarchical topic structure (multiple levels, delimited by slash /) and wildcard ability allows for much more fine-grained filtering at the broker than you may be able to get with a Kafka broker. Consumers only get the messages that they’ve subscribed to, versus having to filter at the receiving end.

Looking to dive deeper into the details? Definitely check out the whole video, and then connect with me here on LinkedIn if you have additional questions.

The post Solace Topics vs. Kafka Topics: What’s the Difference? appeared first on Solace.

Previous Video
Topic vs. Topic: Solace PubSub+ and Apache Kafka
Topic vs. Topic: Solace PubSub+ and Apache Kafka

In this video, Solace dev advocate Aaron Lee explains the differences of how topics are implemented in the ...

Next Video
Ventana Research: Five Reasons to Extend Apache Kafka With An Enterprise Message Broker
Ventana Research: Five Reasons to Extend Apache Kafka With An Enterprise Message Broker