Scaling Microservices with Kubernetes Event-Driven Autoscaler (KEDA) and Solace PubSub+

Event-driven microservices architecture poses unique challenges for scalability, and to realize the full potential of applications and maximize value, the ability to autoscale for capacity is essential. Kubernetes Event-Driven Autoscaler (KEDA) is a tool that helps solve these challenges for event-driven applications deployed on Kubernetes.

Horizontal Scaling of Event-Driven Microservices in KubernetesWhy scaling microservices is important and how Kubernetes and HPA provide an ideal framework for elastic horizontal scaling of microservicesRead the post

What is KEDA?

From the KEDA website, “KEDA is a Kubernetes-based Event Driven Autoscaler. With KEDA, you can drive the scale of any container in Kubernetes based on the number of events needing to be processed.”

What does that mean, exactly? KEDA provides a way to scale event-driven applications based on demand observed from event brokers. KEDA expands the capability of the native Kubernetes Horizontal Pod Autoscaler and is an open source CNCF incubating project (as of this publish date).

When scaling event-driven microservices, the number of messages on a consumer’s input queue is essentially the number of events that need to be processed. Message count, or queue backlog, is an excellent indicator for demand on a given microservice in that:

• If an input queue for a microservice is backed up with lots of messages, we know that demand is high and the microservice should be scaled out.
• If the message count is low (below a defined threshold), then demand is low and the microservice should be scaled down.

With KEDA, you can scale event-driven microservices precisely this way.

KEDA and Solace PubSub+ Event Brokers

Solace PubSub+ Event Brokers provide an ideal backbone for event-driven architecture through their ability to stream events and information across cloud, on-premises, and IoT environments while capitalizing on the full spectrum of event-driven design patterns. In this section I will explain how to scale consumers designed for these patterns that use Solace event brokers.

How does KEDA work with Solace brokers?

Solace PubSub+ Event Brokers have a RESTful API called SEMP that provides access to administrative functions, including read access to queues and their current message counts.

KEDA uses interfaces called scalers to integrate with external event sources. KEDA is packaged with many scalers, allowing it to integrate with different types of technology, including one for Solace PubSub+ Event Broker. The Solace scaler for KEDA integrates with Solace brokers via the SEMP API.

Consider the diagram below. A set of event-driven microservices is deployed to a Kubernetes cluster, each in its own container. Each microservice receives input from a single queue, hosted on a Solace broker. KEDA is configured to monitor queue backlogs using the Solace scaler. Using backlog as a measure for demand, KEDA and the Horizontal Pod Autoscaler (HPA) scale each microservice to the desired replica count.

The diagram depicts the Kubernetes pod replicas being managed based on the backlogs observed for each queue. The queue backlogs provide precisely the information required to scale the applications accurately.

Take a look at “Microservice 2” in the diagram. Note that there are no input messages on the queue, and there are zero active replicas for the microservice. KEDA can scale applications to zero pod replicas, minimizing capacity utilization as much as possible when appropriate. The Kubernetes Horizontal Pod Autoscaler cannot scale to zero replicas by itself, therefore KEDA offers a more efficient solution with the ability to reduce active replicas to zero in situations when there is no demand.

KEDA in Practice: An E-Commerce Retail Example

Let’s examine KEDA in action with an example. Suppose an e-commerce retailer contemplates how to increase traffic to its website and improve its competitive position. The retailer decides to execute a series of “flash sales,” lowering the price of several highly desirable products to generate sales and attract new customers, hoping the experience will lead to repeat business. The customers must have a positive experience with the online store to maximize the likelihood that they’ll return.

The retailer’s order processing architecture is based on event-driven microservices patterns using a cloud-hosted Solace PubSub+ Event Broker. The services comprising the order processing system are deployed to a Kubernetes cluster hosted by a public cloud provider. Orders are received by a front-end “Storefront Service” and processed by a set of backend services: “Inventory”, “Shipping”, and “Billing”.

The resulting architecture and message flow are as follows:

E-Commerce Autoscaling Requirement: Meet Demand Fluctuations

The retailer in this example wishes to manage operational capacity of the backend order processing system to match demand, scaling services out when demand is high and scaling down when demand is low. The retailer expects high demand during the flash sale, but the prospective peak volume is unknown. Some services process more events than others, and others are expected to perform better than others (events/second, for example). Regardless, the entire order processing system must perform to meet demand fluctuations.

The retailer follows good practices for horizontal scalability:

• The microservices are deployed as independent workloads to Kubernetes. Each workload contains a Kubernetes pod that can be replicated for horizontal scalability.
• The event-driven microservices are defined such that the event streams are stateless (no message input to a given service is dependent upon another).
• The microservice input queues are defined as non-exclusive on the Solace PubSub+ Event Broker.

When the platform is under nominal load, such as before and after a flash sale, KEDA maintains lower capacity to match the lower demand. Even so, some of the microservices are less performant. KEDA detects the demand from the event broker, scaling each resource independently. In the example diagram below, the Billing Service is less performant than the other services (perhaps due to a reliance on external transaction processing). KEDA detects the backlog and scales the service automatically based on the demand.

When the flash sale is underway and business is booming, KEDA detects the increased demand and scales the services to meet the demand. Again, some of the queue backlogs are greater than others. But the services are independently scalable and KEDA manages capacity accordingly. In this example, the Shipping Service has more active replicas than the Billing service. The converse was true for nominal demand. Such is the nature of demand: we can’t always predict how systems will react under load. Regardless, KEDA detects the demand from the input queue and makes the scaling decision based on live conditions, not expectations. In this way, the retailer can rely on optimal capacity utilization at a given time regardless of unknowns.

Set Your Platform Up for Success with KEDA and PubSub+

KEDA can be a valuable addition to your event-driven platform. With the Solace Scaler, you can integrate KEDA and Solace PubSub+ event brokers quickly to realize the benefits of accurate event-driven scalability in your state-of-the-art event mesh (network of event brokers).

With KEDA, you can set your platform up for success under varying loads and minimize the risks to system performance from spikes in demand and inaccurate predictions for performance of individual services.

I have included below a set of key take-aways that will be helpful if you want to get started with KEDA and Solace.

• Solace-KEDA Code Lab – Step-by-step instructions to install KEDA and to configure it with a Solace PubSub+ Event Broker. Even if you don’t use Solace PubSub+ event brokers in your architecture, you can complete the code lab in a Kubernetes cluster to get acclimated with KEDA.
• KEDA Lightning Talk – Watch a KEDA core team member discuss KEDA and a demonstration of it in action!
• KEDA Web Site – All things KEDA! Additional technical information, the list of scalers and their associated documentation are here.
• Solace PubSub+ Event Broker Scaler for KEDA – Documentation about the Solace scaler on the KEDA web site.
• KEDA GitHub Project – Contains the KEDA core code, scalers committed to the project, and how you can contribute.

Of course, if you have any questions, please feel free to contact us and we would be happy to help.