Why Horizontally Scaling Message Brokers isn’t All it’s Cracked up to Be

When it comes to increasing the capacity of IT systems you can either scale “vertically” by increasing the power or storage of a given device or piece of software, or “horizontally’ by adding new devices or instances of that software.

One of the biggest problems with increasing the capacity of a software-based messaging platform by adding brokers is the communication that inevitably needs to happen between brokers. When clients or applications connected to one broker need to communicate with an application that’s connected to another broker, you end up sacrificing some of each broker’s capacity to that inter-broker traffic, leaving less available for client communications. In fact, this problem increases exponentially as you add more brokers, so the return you get by adding each new broker is less and less.

The next challenge that IT personnel face when horizontally scaling software-based message brokers is that of capacity planning. With discrete message brokers serving unique sets of clients and instances of applications, each broker must meet the demands placed on it by clients and applications alike with its own available capacity. The complexity of assuring adequate system performance makes load balancing across message brokers much more difficult than load balancing across application instances, and in fact doing so amplifies the inter-broker communications problem described above. This means a given broker can be overwhelmed with traffic (focused overload) while another broker sits idly by with stranded capacity going to waste.

Software-based message brokers can only handle a few thousand messages a second, so capacity problems come up quickly. Solace gives you all the capacity you need (over 200, 000 messages per second per appliance) to meet the need of a few high-volume applications, or lots of low volume applications, all in one compact device with unified administration.

In this 10-minute video, I explain why horizontally scaling message brokers doesn’t work as well as horizontally scaling other assets like applications, and why our high-capacity appliance provides you with a better solution.