Using SEMP for Data Visualization, Provisioning and Monitoring

One of the less well-known but incredibly powerful features of Solace’s Unified Message Platform is Solace Elements Management Protocol, commonly called SEMP. The name says it all — it’s a protocol for managing the various elements of your Solace implementation.

SEMP is pretty intuitive and well documented, and once you get the hang of it you’ll come to think of it as an indispensable tool in many areas of Solace appliance administration. It enables sophisticated scripting and monitoring, and you can even use it to build custom apps for provisioning, visualization, historical reporting and more.

SEMP is actually the foundation of SolAdmin — everything you see on screen in SolAdmin is fetched by SEMP under the covers, which means you can use SEMP to replicate or improve upon anything SolAdmin does in your own applications.

What is SEMP?

SEMP is an XML RPC that works over HTTP or Solace’s message bus.  The Solace appliance has an HTTP listener on a particular port that’s defined during configuration which can receive HTTP requests with XML payload.  For every request Solace sends back an XML response that either delivers requested information or an error if the request was malformed, the sender wasn’t authorized to ask for the info, or the message used a wrong version of the schema.

Sample Request and Response

SEMP Request:

<rpc semp-version='soltr/5_5'>
<show>
<client>
<name>*</name>
</client>
</show>
</rpc>

SEMP Response:

rpc-reply semp-version="soltr/5_5">
<rpc>
<show>
<client>
<client-address>192.168.1.180:58305</client-address>
<name>s0001</name>
<profile>default</profile>
…….. (removed data)
</show>
</rpc>
<execute-result code="ok"/>
</rpc-reply>

Learning SEMP

There are a few ways to get started with SEMP. You could read the SEMP section of the feature guide in your Solace documentation. Or you could examine the SEMP schema files (semp-rpc-reply-soltr.xsd  and semp-rpc-soltr.xsd) that are also part of every release. Load them into any XML editor and you’ll see exactly which requests and responses can be created.

I think the easiest way to get started with SEMP is to enable DEBUG mode in SolAdmin. This will dump lots of information into a SolAdmin log, including all SEMP request/responses it sends to the server. Then just click on any screen and observe which SEMP commands were sent/received by SolAdmin. Copy them into your app and modify as needed. To turn on DEBUG mode in SolADmin, just open log4j.properties and uncomment or add this line:

log4j.category.com.solacesystems=DEBUG,  A1

Sample Applications

Here are some more examples of charts produced using a little bit of SEMP and Google Charts:

A historical reporting tool which shows average latency a VPN experienced over time. Just use a slider at the bottom to observe chart change in real time:

Real Time VPN Usage and Billing Monitor allowing to bill internal departments based on actual usage.

Trade volume distribution across various countries based on SEMP query of various queues that have that info.

High Level Dashboard for Ipad.  Monitor system health from your mobile and see everything at a glance.

Trading VPN latency monitoring

The list of things you can do with SEMP would go on forever.  Anything you can do on a command line or in SolAdmin you can do with SEMP, so in addition to visualization you can add or remove VPNs, add, change or delete users, set up and change access control lists, start and stop services, etc.

But wait, there’s more!

Because when you use it, the requests and responses that carry data go via the control plane and not using any resources on data plane, it has virtually no impact on Solace appliance production performance.  Have your cake and eat it too!
Well almost. We’ll get to it, but one should not get too carried away with marketing slogans, even ones that come from Solace. Though it is true that when you use SolAdmin and CLI tools to monitor production in real time you are not affecting anything on the platform, one need to be cautious on how they use SEMP directly.

Think of SEMP as a low level secret entry into the secret garden normally guarded by higher-level tools like SolAdmin which uses SEMP responsibly.  SolAdmin will not flood the control plane with too many requests that may potentially slow down other monitoring applications that may be also using control plane for other purposes. When you use SEMP yourself, some precaution needs to be taken not to send too many requests for data that does not change that often. A good rule of thumb maybe that if you collect statistics on the box for historical purposes, getting updates every couple of seconds should be MORE than sufficient.

Remember, that other monitoring applications or users may also be using Control plane to do their own monitoring. They maybe running their own copy of SolAdmin to administer their VPNs or doing some other work.  So it is not a good practice to hog a lot of resources (though you got plenty of horse power there) for yourself.

If you require to poll for data more frequently than once a second, do speak with your Solace representative and discuss with them your application needs and what you are trying to achieve. There are many ways to skin the cat they say.  Maybe there is some other functionality of Solace is more appropriate for a task at hand.