Inside a Solace Message, Part 2: Using Header Properties

The first post in this series introduced the basic elements and properties of a Solace message. As explained in that post, header properties are manipulated by API functions. Some header properties are used by Solace message routers when routing messages, the most obvious being Destination.

Application properties are used to help applications process message payloads or to provide a way of exchanging header properties between different messaging protocols – from JMS to Solace Message Format (SMF), for instance.

Other application properties within the header provide a very easy and simple way to add often used data to a message. There is also the option of adding user-defined message header fields to a message.

And, of course, normally you would put the data in the payload- more details on payload encoding in my next post.

This post briefly reviews which header properties are available, and then talks about when you should put data in a header property versus the payload. I will also discuss some of the application properties in more detail and show you how to use user defined message header fields.

Header Properties

Let’s briefly review header properties which are used to route messages. Apart from Destination, Delivery Mode and Reply-To Destination, header properties differ between Direct and Guaranteed messaging:

Direct Messaging Header properties

The header properties relevant to direct messaging are:

• Deliver-To-One: setDeliverToOne(...). Allows load balancing and subscriber redundancy (see this post)
• Class of Service: setCos(...). Prioritize direct message delivery.
• Eliding Eligible: setElidingEligible(...). Deliver messages to clients at a specific rate.
• Cache messages: isCacheMessage(), .getCacheRequestID(), isSuspect(). Associated with PubSub+ Cache request and responses.

Guaranteed Messaging Header Properties

The header properties relevant to guaranteed messaging are:

• Time-To-Live: setTimeToLive(…). This message should expire off a queue after a certain time.
• DMQ Eligible: setDMQEligible(…). This message should be moved to the dead message queue on expiry.
• Immediate Acknowledgement: setAckImmediately().
• Redelivery: getRedelivered(). Delivery of this message has been attempted before.

Adding application properties to a message

Application properties are used to exchange information between applications. They are part of the message header but don’t affect messaging routing – they are passed through the message router.  Application properties are accessed in the same way as header properties, using getters, setters and sometimes is functions:

The following is a list of the application properties available – they’re fairly self-explanatory:-

• getReceiveTimeStamp()
• setSequenceNumber()
• setSenderId()
• setApplicationMessageId() for compatibility with JMS (ApplicationMessageId maps to JMSMessageId)
• setApplicationMessageType() maps to the JMS property, JMSType.
• setHTTPContentType()
• setHTTPContentEncoding() Used by the Solace REST Messaging API when mapping HTTP message content types to and from SMF message types – see the REST Messaging Protocol Guide)

Some of these properties can be populated automatically (e.g. Time Stamps and Sender IDs – see the relevant session properties to enable this automatic population), but you can overwrite the value. You lose the automatic population if you set the value manually, so remember to change the value should you send the same message more than once.

Using these methods is simple. Once you’ve acquired your message using JCSMPFactory.onlyInstance().createMessage(…), simply call the method:

TextMessage textMsg =
    JCSMPFactory.onlyInstance().createMessage(TextMessage.class);
textMsg.setSequenceNumber(0);

User Data

Solace messages can be optionally populated with User Data. This is a variable length field of up to 36 bytes. User Data is unstructured and offers fast access with low overhead. It is perfect when you need to provide a small amount of data to an application very fast – for instance, a hint on the payload contents.

Where should I put my data?

It’s worth taking some time considering whether to use these application properties, user defined header fields, user data or encode the information in the payload itself.

It’s worth remembering that while header fields can be used with Selectors, in general we don’t recommend using selectors to filter messages. Most of the filtering that can be achieved with selectors can be achieved with topic routing. Topic routing offers better performance and is more portable between protocols.

All APIs support header fields with the exception of MQTT, because the MQTT specification dictates what can be provided in the header.

Use the API provided function if:

• You need an easy way to access the data;
• You’d like to use Selectors on the properties;

The API functions have fixed property types.  If you need a different format for the data, say you’d like to use a different date type for the SenderTimeStamp, you could use a user defined header field.

Use a user defined header field if:

• You need to provide a fine-grained or rarely-used hint to the receiving applications about the content of the message. Coarse grained information, for instance data format versions, are probably best off in the topic;
• You need a different data type in your header field from that provided by an API function;
• You have a well-structured, common requirement for a simpler data type;

Use message User Data if:

• You need fast, low overhead access to a small amount of data (36 Bytes);
• The data does not need to be structured and is traversing a homogeneous network.

Encode this data in the payload if:

• It’s normal message data – this is the natural place for message data;
• The hint you’d like to give to the receiving application isn’t a simple key-value pair. Situations like this are dealt with in the next post;
• You would like to use an external source for the data format, for instance Avro or a Google Protocol Buffer. This allows organization wide distribution and versioning of the message format, independently of the messaging code;
• You must ensure maximum portability or be vendor agnostic – for instance, the MQTT 3.1 protocol does not support user defined header fields (5.0 does).

I’ll talk about the options for encoding payload data in my next post, Inside a Solace Message, Part 3: Payload Data.

Moving messages between protocols

Solace message routers translate between messaging protocols: you might publish a JMS message, but this can be received by REST, MQTT and SMF clients.

Different messaging protocols have different header fields.  For instance, HTTP has the content-type and content-encoding fields; these make very little sense to a JMS application, but when the router is about to send a REST message it needs to know what you want to put in these header fields.

Mapping between JMS & SMF.

Some JMS Properties map very obviously to Solace message header properties:

• JMSDeliveryMode <-> DeliveryMode
• JMSMessageID <-> AppliationMessageId
• JMSTimestamp <-> SenderTimestamp
• JMSCorrelationID <-> CorrelationId
• A full list can be found in the JMS API documentation

For instance, if I send message from Java:

sendMessage.setApplicationMessageId(“MY_JMS_Message_ID”);

The message will be received with a JMSMessageID in JMS:

String msgId = receivedMessage.getJMSMessageID();

Mapping between REST & SMF

The Solace messaging platform offers an HTTP REST messaging interface which is pretty unique in that it can queue and persist incoming REST messages and offers Solace levels of performance (in other words, the best you can get). You also get full, transparent interoperability with the Solace messaging platform. You can send in a REST request and receive the request in JMS, MQTT or any other supported Solace protocol without any need for bridges or adaptors.

It is worth remembering that there are many differences between header fields in HTTP REST messages and other protocols. You can find a full description in the REST Messaging Protocol Guide.

An example of Application Header use is HTTP Content-type. When sending message to a Solace message router, the Content-Type will dictate what type of Solace message is created. If you’re sending a content type “text, ” this will result in a TextMessage to JCSMP.  However, when sending to a REST consumer, in some cases, you’ll need to provide more hints to the Solace message router so that the REST message contains the desired Content-Type. If you want to send JSON from JCSMP to a REST consumer, for instance, and the receiving application can process either plain text or JSON, you can provide this hint using HTTPContentType:

Message.setHTTPContentType(“application/json”);
Message.setText(jsonString);

The REST consumer will receive an HTTP POST with a Content-Type of application/json.

Mapping between MQTT & SMF

MQTT is a deliberately lightweight protocol – there are only 7 fields in an MQTT 3.1 publish packet header, for instance.

An MQTT message published to a Solace message router will have default values used for mandatory properties for messages forwarded to non-MQTT clients. In the other direction, most of the header properties have to be stripped before sending to an MQTT client. See the MQTT Specification Conformance guide for more details.

User Defined Message Header Fields in Action

User defined message header fields use Solace Structured Data Types (SDTs) in a User Property Map. This is a key value pair. The data types are guaranteed to traverse heterogeneous networks, and the key/values can be used with Selectors. They are a simple, easy and convenient way to add simple data to messages.

Creating the User Property Map

The first step is to create a container for the User Properties, the Map. As its name implies, this is a key/value mapping. In Java, the Map can be created independently of messages, so it can be used in multiple messages very easily. Neither ordering nor uniqueness is guaranteed in maps. In the Java API duplicate keys are detected, but not in C or .Net.

We use the factory to create the map container:
SDTMap map = JSCMPFactory.onlyInstance().createMap();

Adding fields to the Map

Once we have our map it’s a simple matter to put your key/value pair to the map:

map.putBoolean(“MyKeyIsPresent”, True);

On reception of the message, it’s just a matter of getting the property:
myKeyIsPresent = map.getBoolean(“MyKeyIsPresent”);

Mapping between the various Solace SDTs between different languages is shown in the documentation for using structured data – for instance, if you put a java.lang.String, you’ll get a const char * in C and a System.string in .NET.

Attaching the map as User Properties.

It’s a simple matter now to attach this user map as a Property Map to the message:
Message.setProperties(map);

And we’re done.  In this example we’ve created a boolean User Defined Property called MyKeyIsPresent.

Getting the map

On reception, we can simply get the map, and then get any key in it:

Message.getProperties().getBoolean(“MyKeyIsPresent”);

Obviously you’ll need to surround this with try/catch to handle the exception if the key isn’t present. If the map isn’t present, getProperties() returns null.

User Defined Properties must be embedded in a map, but one of the SDT types is a map, allowing nesting.  Another SDT type is a stream, which is a sequence of structured data types.  Ordering is guaranteed in a stream, of course, and access is faster than a map, but must be sequential.

It’s possible to iterate through your SDTs with access functions such as isEmpty(), keySet(), containsKey(), containsValue(), size(), values(), hasRemaining() and read(), allowing access to SDTs of unknown structure.

SDTs in other protocols

In REST, flat SDT types are mapped to HTTP header properties:

'solace-user-property-MyKeyIsPresent’: 'true; type=bool'

MQTT 3.1 doesn’t have the ability to send User Defined Properties or receive them. There is no provision in MQTT 3.1 to extend headers or a way of encoding them in the payload, so they are simply dropped. MQTT 5.0 has been updated to provide user properties, including provision for request/response.

Summary

Message headers such as Destination are used to route messages. Also present in the header are application properties allowing you to provide a hint to the receiving application as to how to deal with the message payload.

You can also use Application Header Properties as a quick and easy way to exchange simple data.  If you need more flexibility in the data to be exchanged, for instance there’s no suitable Application Header Property for your needs, a user defined header field has the advantage of transparency across architectures while remaining easy to use.

If you need to exchange more complicated data, for instance something that doesn’t match a simple key/value pair, you’ll need to encode the data in the payload – and that’s the subject of the next post in this series, Inside a Solace Message, Part 3: Payload Data.