Mapping AMQP 0-9-1 Semantics onto Publish/Subscribe Messaging

Have you ever needed to map AMQP 0-9-1 semantics onto pub/sub style messaging? Or wondered if you could even do that?

Those of you working with AMQP know that versions pre-dating 1.0 describe a very different messaging style than most messaging systems. AMQP 0-9-1 (which predates the Oasis AMQP 1.0 standard,  is used by RabbitMQ, and is the model I’ll be referring to here) revolves around exchanges and queues. Every message sent is being published to an exchange, and every message received is being delivered from a queue. In order to have any messages forwarded to queues, you have to bind them to the exchange that you want to receive messages from.

AMQP 0-9-1 also knows different types of exchanges, which influences how messages are being forwarded from exchanges to queues. There’s direct, fan-out, topic and header exchanges, plus you can write your own plug-in exchange with a custom routing algorithm. If you are interested in learning more about the various types you can find a nice introduction on the RabbitMQ web page.

How is that so different from other messaging styles? In a JMS style broker, for instance, you can send and receive messages straight to and from queues (point-to-point) or send and receive directly from topics (pub/sub). You can also send messages to topics and receive them from queues, and some providers even let you do it the other way around!

Migrating from an 0-9-1-style Broker to Solace

So what if you want to move from an AMQP 0-9-1-style broker like Rabbit to a Solace Message Router? How would you map the various type of exchanges to the messaging styles available on Solace?

The key is to use a mix of pub/sub and point-to-point, and map exchanges onto the topic namespace while using queues to subscribe to them, just like how queues bind to exchanges in AMQP.

The trick is mapping the different exchange types onto the topic namespace. So here’s how I approached it. You need to decide whether or not to include the exchange type in the topic naming convention. I see two options for the topic name prefix:

1. <exchange name> as the top level topic
   The advantage of this approach is publishers and consumers don’t necessarily need to know the type of exchange they are communicating with. This abstracts away from the application, and some of your applications may already make use of this. Exchanges could be declared by other applications or administrators in this case. The downside is that you have to watch out for namespace clashes as you could use the same exchange name with a contradicting type leading to unpredictable behaviour. You should choose this option if your applications need to be able to use exchanges without knowing their type and you can manage the exchange name space to avoid clashes. You can also isolate any clashing applications by putting them on different message VPNs in Solace.

2. <exchange type>/<exchange name> as the top two topic levels
   This approach eliminates any type clashes with your exchanges, but it comes with the price that all the publishers need to know what type of exchange they are publishing to as they have to give the full name and can’t use wildcards. Only your consumers would be able to subscribe to the exchange agnostic of its type by using a wildcard on the top level. You should choose this option, if your applications are already aware of the exchange type they are using or can easily be changed.

I like the first option as it seems to be a good idea to keep the routing logic abstracted away from the applications.

Schema

Once you’ve decided on the top level topic naming convention, you have to think about how to implement the various exchange types. I applied the schema described in the following section. I will refer to the top level topic prefix by <topic prefix> going forward, which can be substituted by either <exchange type>/<exchange name> or just <exchange name> depending on your choice.

Fan-Out Exchanges

Fan-out exchanges are simple top level topics, any additional parameters should be ignored. In order to achieve this, subscriptions that bind queues to these topics need to subscribe to everything under the top level using a wild card. Solace requires two subscriptions, one for the top level (e.g. “<topic prefix>“) and a second one to capture the sub-levels, e.g. “<topic prefix>/>” where the last > is Solace’s multi-level wildcard. Ideally the publisher also removes any routing keys when publishing to suppress any information about routing keys being passed down to the receiver, but it’s not necessary from a routing perspective. Let’s look at an AMQP application a news agency might use to publish news to customers. The application could be publishing messages to a fan-out exchange:

• AMQP exchange name=“world_news” exchange type=”fan-out”

When moving the application onto Solace, we would map this onto a topic:

• Solace topic=“world_news”

Direct Exchanges

Direct exchanges allow for publishing a routing key, which receivers can filter for. The routing key becomes the second level in our topic hierarchy appended to the prefix:

• “<topic prefix>/<routing key>”

In our news agency example, we could imagine a direct exchange called “regional_news” and messages published to this exchange will contain routing keys like america, europe, africa, asia and australia. This can be mapped as follows:

• AMQP exchange name=“world_news”, exchange type=”direct” with routing key=”america”

Moving the application onto Solace, we would map this onto:

• Solace topic=“world_news/america”

Topic Exchanges

Topic exchanges are more similar to traditional pub/sub messaging and they allow the use of topic strings including a hierarchy. When mapping this onto Solace the actual topic string will start at the second level following the exchange name at the first level. In the original AMQP topic string the topic level separator “.” (dot) needs to be replaced by “/” (forward slash).

If our news agency wants to be a bit more specific about the type of news it’s publishing, we could imagine our previous “regional_news” exchange being turned into a topic exchange and messages published to this exchange will contain routing keys with a structure like america.canada.ontario. We would map this as follows:

• AMQP exchange name=“world_news”, exchange type=”topic” with topic string=”canada.ontario”.

Moving the application onto Solace, we would map this onto:

• Solace topic=“world_news/america/canada/ontario”.

Header Exchanges

Header exchanges are similar to fan-out exchanges in the sense that they also ignore any routing keys. But the distinguishing factor is that they can filter on properties of the message headers that are being published to them. This means that we can use the same naming pattern as for our fan-out topics, but the queue bindings are being translated into topic subscriptions with filters on them that only forward the messages to queues, which match those filters.

If we want to turn our “world_news” exchange into a headers exchange, we would see those messages being published onto:

• AMQP exchange name=“world_news”, exchange type=”headers” with a binding filter.
• Solace topic=“world_news” with a selector.

Then we can imagine applying filters/selectors to these messages to only get messages with headers matching something like category=’technology’ and subject=’messaging’.

Queue Binding

Lastly, a queue binding is being translated into a queue subscription replacing any wildcard characters as appropriate. In AMQP all queues have a special feature as in that they are also automatically bound to the default exchange called “” (empty string), which is a direct exchange and the queue name is used as routing key. As mentioned earlier, you cannot publish directly to queues in AMQP. Instead messages are sent to the default exchange with the queue name as routing key. We could attempt to replicate that behaviour by publishing messages to a topic like “/<queue name>”, but this would present an empty top level and Solace doesn’t permit that. But we can instead just send messages directly to queues in Solace.

If you are using the more advanced feature of tapping into messages destined for certain queues via the default exchange, then you will be interested to hear that this is also possible in Solace. Every queue subscribes to a special topic following the pattern “#P2P/QUE/<queue name>”, which you can add subscriptions to as all messages destined for that queue are published to it.

To summarise, the naming pattern for topics described above roughly translates into these expressions:

• <Exchange name>/[<Routing key> | <Topic string>]

Or (if you have chosen option 2 for your topic prefix):

• <Exchange type>/<Exchange name>/[<Routing key> | <Topic string>]

Although it’s difficult to get a 100% match of functionality and behaviour, this approach will let you replicate most AMQP 0-9-1 behaviour on a Solace Message Router.

If you’ve developed an approach that works even better, please share in the comments.