Understanding Guaranteed Message Publish Window Sizes and Acknowledgments

One of the main advantages of Guaranteed Messaging (a.k.a.  Persistent Messaging or store-and-forward messaging), is that a receiver does not have to be available for the publisher to send messages as the Solace Message Router will persist the messages until the receiver becomes available again. The publisher is guaranteed that a message is successfully persisted by the Solace Message Router once it receives an acknowledgement for the message it sent. Persistence increases reliability but does add some overhead compared to Solace’s Direct Messaging, which provides no acknowledgments to the publisher.

Despite this overhead and the round trip times of acknowledgements, Solace Message Routers provide a high throughput rate for Guaranteed Messaging publishers. This high performance is enabled in the SMF protocol through the use of a Publish Window. In this post, I will introduce you to the concept of Publish Windows and their sizes, and how acknowledgements affect these windows. We will also look at when and how to adjust the sizes of the Publish Window using Solace’s Java API.

Understanding Publisher Windows in Guaranteed Messaging

When an application publishes messages using Persistent delivery mode, the Publish Window will determine the maximum number of messages the application can send before the Solace API must receive an acknowledgment from the Solace Message Router. The size of the window defines the maximum number of messages outstanding on the wire. So if my Publish Window size is 10 that means my application can publish a maximum no of 10 messages before the Solace API must receive an acknowledgment from the Solace message router. Once an acknowledgement is received the window opens up and the application can send more messages. This is very similar to how Sliding Windows work in TCP’s Flow Control protocol.

To help us understand this and how the window opens up when acknowledgments are received let’s take a look at the basics of windowed publishing in general using the following sequence diagram.

Consider in the above example the Publish Window size is set to 5. The publisher won’t be able to send the 6^th message and will be blocked on the send call, until it has received the acknowledgment for the previous messages. As soon as it receives the acknowledgement the window opens up and it can accept and publish more messages from the application.

It would be very inefficient if the application was constantly waiting on the send call for the Publish Window to open up. The Solace Message Router is much smarter here and instead it sends an acknowledgment for published messages when a threshold of either a third of the Publish Window size is reached or fixed timeout of one second has elapsed this allows messages to flow at very high rates. Let’s look at another example to understand how and when the Solace message router sends these acknowledgments to the Solace API.

Consider in our above example the Publish Windows size is 12. The Solace Message Router will send back a single ACK to the Solace API if it has persisted one third of the messages in the Publish Window. This single ACK from the Solace message router implicitly acknowledges all previous messages (message 1-4 in our examples above).

As seen in the diagram above the API has accepted and published messages 1-5 from the application and received an ACK for the first 4 messages, so at this point of time our Publish Window has opened up and now it can accept 11 more messages before it must receive another ACK from the Solace message router. This is similar to the Sliding Window mechanism used by a number of protocols today, such as TCP, and allows the publisher to send Guaranteed Messages at a much higher throughput, without having a lot of published messages pending to be acknowledged.

But what if my application only published 3 messages in the above example? As mentioned previously the Solace message router sends the acknowledgment if either a third of the Publish Window size is reached or one second has elapsed. So if the 4^th message is not sent and one second has elapsed, then the Solace message router will acknowledge the last three messages it had persisted. Both the one third threshold and the one second timer are non-configurable properties on the Solace Message Router and have been optimized to provide the best message throughput.

Message Rejection

It is possible that a message may be rejected and discarded by the Solace Message Router. For example, if my application does not have the right ACL permissions to send the Guaranteed message, then the message would be discarded. In such cases, the Solace Message Router sends back to the publisher a message rejection, a.k.a. negative acknowledgment, indicating the message could not be persisted and was discarded. Refer to the “Enabling or Disabling Reject Message To Sender on Discard” section of the Solace Feature Provision Guide to see the possible scenarios when Solace Message Router would send a message rejection.

Message rejections also affects Publish Windows. Say for example my Publish Window is closed and the Solace API receives a message rejection, then the Publish Window would open up to accept and publish one more message.

Handling Guaranteed Message Acknowledgments

Now that we know how messages are sent by the Solace API inside a Publish Window, and how the window is affected when acknowledgments and message rejections are received from the Solace Message Router, let’s see how the application can handle these acknowledgments.

When an application receives an acknowledgment for a message it published, it can be guaranteed that the messages has been successfully handled and persisted by the Solace Message Router and the message won’t be lost. Previously we had seen that the Solace Message Router can send a single acknowledgment to the Solace API for a range of messages in the Publisher Window. The application has two options on how it wants to receive these acknowledgments from the Solace API:

1. Per Message Acknowledgment Event Mode (default)
2. Windowed Acknowledgment Event Mode

In the Per Message Acknowledgment Event Mode, which is the default mode, the Solace API will provide the publisher with an acknowledgement for each message it published. As illustrated by the sequence diagram below, even if the Solace API receives a single acknowledgment from the Solace Message Router it will provide the publisher with explicit acknowledgment for each message it sent.

In the Windowed Acknowledgment Event Mode a single acknowledgement event acknowledges the last message published and implicitly acknowledges all of the preceding messages, as illustrated by the sequence diagram below.

In both Acknowledgment Event Modes, message rejection remains explicit. That is, the publisher receives an explicit message rejection for each published message that was rejected by the Solace Message Router. So in our above Windowed Acknowledgment Event Mode example, say if msg3 was rejected then, the Solace API and publisher will first receive a single windowed acknowledgment for msg1 and msg2, then a message rejection for msg3 and a single windowed acknowledgment for the other persisted messages, as shown in the sequence diagram below.

When and How to Modify Publish Window Sizes and Ack Event Modes

The default Publish Window Size configured by the Solace APIs have been proven to provide good performance for most LAN situations when applications are within the same data center, with the exception to the Solace Java API, which uses a Publish Window Size of 1 due to backward compatibility reasons. Therefore, it makes sense to always set the Publish Window Size to 50 by default in your applications, which is also the default configured value in other Solace APIs.

But what if your application is residing in a different data center connecting over a WAN link, or your producer is on LAN network with higher delay (possibly incurring router/switch induced latencies), and the underlying TCP connection is experiencing flow control situations? In such cases you might want to tune the Publish Window Size to best suit your messaging environment. A Publish Window Size set to 255 would be a good value to begin with in a network with higher latencies. When choosing a value, you should take into consideration the network latency between your publisher and the Solace Message Router, and tune to a value that provides the best throughput rates in your network.

The Publish Window Size can be adjusted via Session properties in the Solace API. Below is an example to adjust the Publish Window Size value in the Solace Java API to match that of the other Solace APIs like C and .NET.

final JCSMPProperties properties = new JCSMPProperties();

/* Other required connection properties omitted here */

properties.setProperty(JCSMPProperties.PUB_ACK_WINDOW_SIZE,  50);

final JCSMPSession session = JCSMPFactory.onlyInstance().createSession(properties);
session.connect();

The above sets the Publish Window Size using the session property JCSMPProperties.PUB_ACK_WINDOW_SIZE to 50, which is the default used by Solace C & .NET APIs, and recommended value for the Solace Java API. JCSMPProperties is passed as a parameter to the createSession method when creating a new Session. To adjust this window size in other Solace APIs refer to Adjusting the Guaranteed Message Publish Window Size in the Solace Messaging APIs Developer Guide.

You can also configure your publisher to use a Windowed Acknowledgment Event Mode, which we discussed in the previous sections. Using a Windowed Acknowledgment Event Mode is useful when you want to process a single event acknowledging a batch of published messages, instead of processing individual acknowledgments, which could also help improve performance of handling acknowledgments.

The Acknowledgment Event Mode is set via session properties similar to the Publish Window Size. In the Solace Java API add the following code to the above JCSMPProperties to use a Windowed Acknowledgment Mode.

properties.setProperty(JCSMPProperties.ACK_EVENT_MODE,  JCSMPProperties.SUPPORTED_ACK_EVENT_MODE_WINDOWED);

To adjust this mode in other Solace APIs refer to Setting a Message Acknowledgment Event Mode in the Solace Messaging APIs Developer Guide.

Putting it All Together

The complete source code can downloaded from here PublishWindowSize.java. This sample builds on the source code provided with the Confirmed Delivery (Java) tutorial. The Java API library can be downloaded here. The Java API is distributed as a zip file containing the required jars, API documentation, and examples. These instructions assume you have unpacked the Solace Java API into a directory next to the PublishWindowSize sample code that you just downloaded. If your environment differs then adjust the build instructions appropriately.

Building this example is simple. The following provides an example using Linux. There are many suitable ways to build and execute these samples in Java. Adapt these instructions to suit your needs depending on your environment.

In the following example replace VERSION with the Solace API version you downloaded.

javac -cp sol-jcsmp-VERSION/lib/*:. PublishWindowSize.java

Run the example from the command line as follows.

java -cp sol-jcsmp-VERSION/lib/*:. PublishWindowSize HOST

Summarizing

I have shown you how Solace APIs use a Publish Window to send messages to the Solace Message Router while focusing on the Solace Java API for the examples. I have also shown how the size of this window limits the number of messages the publishing application can send before the Solace API must receive an acknowledgment. We learned how this window is affected as acknowledgments and message rejections are received by the Solace API.

The default Publish Window Size defined by the Solace APIs have been proven to provide good performance, and can be adjusted to better suit your messaging environment for ex. when sending messages over the WAN. I also showed you how to adjust this window size and the Acknowledgment Event Mode using the Solace Java API.

More details on Publish Window Size and acknowledgment can be found in the Publishing Guaranteed Messages section of the Solace Messaging APIs Developer Guide.

Now hopefully you can apply these concepts to write more effective applications that maximize the publisher throughput in your environment.