Any sizing exercise with a joint Boomi/Solace solution is going to start with this question, but Solace connections and Boomi connections aren’t the same, so it’s important to understand what the word means in each environment.
Boomi counts each unique link to an application, database, event broker, etc., as a connection. But it’s really a measure of supported login credentials, not a network connection. If you want to connect to a database, you need to specify username, password, IP address, port, and database. Each unique combination of those data constitutes a Boomi connection that needs to be licensed, but you can use that set of credentials to create multiple concurrent connections to the same database (same username, same password, etc…) while only paying for a single Boomi connection.
Solace takes a more network-layer approach and considers every single socket connection between a broker and a client as a connection. That means a Boomi Atom that uses the same connection object (credentials) to create 10 concurrent connections to a Solace broker is using 10 Solace connections, but only 1 Boomi connection.
To support concurrent connections to a Solace queue, in order to enable parallel processing (in Boomi) of data from that queue, you need to ensure that the queue is set to support Non-Exclusive queue bindings. By default, the Solace connector creates queues with Exclusive bindings, which means each concurrent connection after the first would be treated as a failover application and only receive data if the client connection created before it was to fail or be closed. I’ll explain how to do that below.
There are a few ways Boomi users can tune performance, such as using parallel processing, to increase throughput. The first two are connector operation configuration settings, MaxConcurrentConnectionCount and batch, and the third is Boomi Molecules.
The table below shows the number of Solace connections associated with different variations of the above.
The above table assumes that MaxConcurrentConnectionCount is the same value across all Boomi connection operation configurations. Boomi allows you to set these differently, and it would change the total number of Solace connections. For example, if the # of Boomi connections is 5 and only 3 of the process operations has a MaxConcurrentConnectionCount of 2 with the others being set to 1 and there were still 3 nodes, the Solace connections would be (2+2+2+1+1) connections x 3 nodes = 24 Solace connections, instead of the 30 in the table above when all 5 Boomi connections have 2 concurrent connections.
In the Boomi Atom Cloud, Boomi has implemented the concept of Atom Workers, again for performance enhancement. The Boomi Atom Cloud is a multi-tenant environment, so it is possible (but not common) to have resource contention across different end customers. You can think of Atom Workers as multiple instances of a given process, but they are reserved for use by the given customer Atom so there is no resource constraint and no startup time (analogous to doing a warmup of a JVM prior to performance testing).
These do appear to be similar in function, but they’re implemented quite differently. The main difference is where you use them:
Solace connection behavior with Atom Workers is the same as behavior with Boomi Molecule Nodes and connector settings:
This table shows some examples of how many Solace connections would be associated with different variations of the above.
Similar to the table for Boomi Molecules, there is an assumption that all Boomi connection operations in the table have the same setting for MaxConcurrentConnectionCount. The math would be the same as above should there be different settings for different connection operations.
A licensed Boomi connection is a set of credentials in a Boomi connection object that is deployed. It does not need to be a listener as producers are Boomi connections and not listeners. But you can have multiple instances of the same connection credentials: MaxConcurrentConnectionCount, Molecule Nodes and Atom Workers all instantiate additional instances of a single Boomi connection. These all still count as a single Boomi connection, despite their manifesting as multiple client connections to Solace.
On the Solace side, each client consumer counts as a Solace connection. You could have 100 client consumers bound to Solace with the same credentials, from a single Boomi connection object. That is 1 Boomi connection and 100 Solace connections.
You can use Solace’s WebUI to confirm and track work done by concurrent connections from Boomi.
Solace WebUI –> Queues -> Select your queue -> Consumers
Here you can view all current client connections, the unacknowledged messages being processed, and the messages confirmed as processing completes.
If you decide to test the above, you will note quickly that the distribution may not appear to be distributed evenly across the Atom Workers or Molecule nodes, especially when the volume is high. Solace is supposed to use a round robin distribution across non-exclusively bound connections. I found often that across a set of 3 nodes, or Atom Workers, 1 node would get 90% of the load and the others the remainder, even if the batch size on the operation was set to 1.
This is because Solace queues will continue to push events to consumers while the consumer has not acknowledged successful processing of the first event. As that message is being processed by Boomi, it is considered a “delivered but unacknowledged message”. Solace will continue to push events to the consumer until a certain number of unacknowledged messages have been sent. That limit is driven by the queue setting: Maximum Delivered Unacknowledged Messages per Flow. When that limit is reached, Solace stops sending events. Once an acknowledgement is received, events can start flowing until Maximum Delivered Unacknowledged Messages is reached again.
What this does is determine how many events a flow (think of this as a Solace connection) can consume while it still has others it has not completed processing yet. Default is 1,000, which may be too high to allow for even distribution across all the connections, so you may want to tune this to a lower level based on your message rate. For example, for a total event message volume of 100, I set mine to 10. See below how to do this.
Here’s the two simple steps it takes to set the number of delivered unacknowledged messages a given flow will allow.
If you aren’t familiar with the different kinds of Solace endpoints and how to configure them, I suggest you read my colleague Leah’s excellent post about that.
And just like that, you’re all set!
I hope this post has helped you understand how Boomi and Solace each define connections, and how to figure out how many Solace connections you need to support a given number of Boomi connections.
Subscribe to Our Blog
Get the latest trends, solutions, and insights into the event-driven future every week.
By submitting this form, you agree to Solace’s privacy policy: solace.com/privacy-policy/