Entries by Tom Fairbairn

How to Maximize Microservices by Combining Messaging, REST and Webhooks

One of the real pleasures of my job is working with customers to understand their use cases, design decisions and architecture. Recently I’ve been having a lot of conversations about how to maximize microservices so that they communicate with each other – it’s a fascinating process that usually starts with REST and goes down some interesting roads when real-world requirements and constraints come into play.

For example, one of my customers was leveraging a microservices architecture to implement persistent request-response style interactions from their customer-facing user interface (UI) into their CRM/ERP system and database layer. It sounds simple: a typical enterprise integration pattern for intra-process communications, but, of course, these things never are.

In this case their UI used REST, the CRM/ERP system relied on a PHP module to interface with the outside world using a Webhook-style integration, and between them sat a validation and enrichment module that had been written in Node.js.… Read the rest

Blockchain-Based Ledgers are Reshaping the Financial Back-Office

During any kind of financial transaction, before the trade, both parties must arrive at a shared understanding of market conditions and prices. After the trade, they need a mutually agreed upon means of processing, recording and settling the transaction. Each firm brings to the table their own applications, algorithms and infrastructure for exchanging data, and together they must agree on how to execute the transaction.

Today, many financial institutions hire 3rd-party settlement agencies (like CLS) to ensure the integrity of these transactions, trusting them to identify and rectify situations where things go sideways, whether innocently or as the result of fraudulent or non-compliant activity. Each company has a team that deals with reconciling these discrepancies. Including these agencies in the high-volume flow of transactions is important, but adds considerable complexity to the flow of information, and can be quite expensive.

The opportunity to fix this glaring market inefficiency is what has so many people in the financial services industry excited about blockchain-based ledgers: a single source of truth that all counterparties can see and trust.… Read the rest

Getting started with the SolOS CLI

Most people starting with Solace will probably use our administration GUI, SolAdmin. SolAdmin is an excellent tool, but as with any GUI, once you get used to the concepts and procedures of using Solace message routers and gain experience, you may want to use a Command Line Interface (CLI) for some tasks.

Using the SolOS CLI has some advantages:

  • Its structure closely matches that of the SEMP management interfaces and associated APIs, so those familiar with the CLI will find programmatic management easier;
  • It is possible to create replay files with CLI, which can automate tasks – such as moving a Message VPN configuration from test through UAT to production environments;
  • For some tasks, the CLI is faster.

I personally use a mixture of the CLI and SolAdmin. I find VPN bridges tricky in the CLI but a cinch with SolAdmin, while I’d rather use the CLI for redundancy configuration and initial setup. … Read the rest

Buffing Up Your Messages: Making Use Of 3rd-Party Data Serialization Systems

In previous blog posts I’ve introduced three ways of encoding message data: raw binary in the payload, some form of structured text such as JSON or structured data types.  The last option I’d like to cover is making use of a third party data serialization system such as Apache Avro or Google Protocol Buffers (GPB).

These third party serialization systems offer some advantages that aren’t available with any of the other options: they give you an independent source for the structure of data.  This source can be versioned and controlled just like any other source, but the changes are localized to the structure of the data, so changes can be made without changing or breaking any other code. Accessing the data is abstracted, providing a layer of automation which reduces the chances of errors in data access.  For any large project with multiple developers using the same data format, this single source of data structure can have considerable benefits.… Read the rest

What a Load of JSON: XML and other Text Data in The Solace APIs

I recently wrote a series of blog posts that introduced the elements of Solace messages including message header properties and how to encode data in the payload.  Now I’d like to dig a little deeper and talk about using text messages in Solace. Text messages offer more than just simple string transfer: common formats such as JSON and XML allow the transfer of complex data as strings with readily available libraries to write, read and parse the data for you.… Read the rest

Inside a Solace Message, Part 3: Payload Data

solace-message_payloadIn the first post of this series I introduced the structure of Solace messages, and in part two I talked about how Application Header and User Defined Properties can be used to decorate messages to aid processing the message data.  Now it’s time to talk about how the data itself is structured – how to put the payload in the message.

Don’t I Just Add the Data?

At first glance this question seems trivial: don’t I just put the data in the message?  That’s the point of a data agnostic messaging system, surely? Well, yes… and no.  I hope you weren’t expecting a different answer!

Solace Message Routers will transport whatever data you choose to put in your message (provided it fits within the maximum message sizes, 64MB for direct and 30MB for guaranteed)  But there are other considerations:

  • Is your network heterogeneous? For instance, are mobile devices part of your messaging infrastructure? 

Read the rest

Inside a Solace Message, Part 2: Using Header Properties

solace-message-emphasis-headerThe first post in this series introduced the basic elements and properties of a Solace message. As explained in that post,  Header Properties are manipulated by API functions.  Some header properties are used by Solace message routers when routing messages, the most obvious being Destination.

Application properties are used to help applications process message payloads or to provide a way of exchanging header properties between different messaging protocols – from JMS to Solace Message Format (SMF), for instance.

Other application properties within the header provide a very easy and simple way to add often used data to a message.  There is also the option of adding user-defined message header fields to a message.

And, of course, normally you would put the data in the payload- more details on payload encoding in my next post.

This post briefly reviews which header properties are available, and then talks about when you should put data in a header property versus the payload. … Read the rest

Inside a Solace Message, Part 1: An Introduction

It’s a fairly obvious statement that a message contains a header and a payload, the header being used to direct a message to its destination(s) with the payload being the data to be consumed.

Some message formats (for instance Solace Message Format, SMF) allow user properties to be added to the message, along with the header, or for structured data types to be added to the message.  When should these be used, and how?  How should the payload be encoded?  As a developer, how do I access message properties?

Message Format Support in Solace

  • REST: HTTP style messaging with a twist: persistence!
  • MQTT: Lightweight wire-line protocol gaining acceptance in IoT;
  • JMS: enterprise, open Java API for messaging;
  • OpenMAMA: an open API for capital market data systems;
  • SMF: Solace Message Format, Solace’s proprietary format that is the canonical format on the Solace platform;
  • Web Messaging: A Solace proprietary format for  web browser messaging

This series of blog posts will examine some of these details, looking at message structure, user-defined header properties, options for payload encoding with things like JSON, Google Protocol Buffers and Solace options like structured data types. … Read the rest

When Topics Aren’t Just Publish/Subscribe: Explaining Topics & Subscriptions in Solace

I was at a retail customer recently helping them migrate from their existing infrastructure. Their message brokers have run out of capacity, so they are evaluating how Solace could simplify their messaging environment by providing all the scale they need in a single easy to manage appliance.

Things were going well when I was asked a question I often hear: “OK, so how do I create a Topic?”

This customer understood a “topic” to be an object on the message broker, created administratively and able to persist messages: in JMS terminology, he understood a “topic” to mean something like a Durable Subscription. (in Solace we map this to a Durable Topic Endpoint which is the broker artifact that persists messages).  Note that a JMS topic is not the same as a durable subscription –  unfortunately the same terms in messaging can have widely differing meanings between similar products.

This lead to the question “what is a Topic in Solace?”  The answer is surprisingly simple: “In Solace, a Topic is a property of a message.” That’s it. … Read the rest