Kernel bypass – revving up linux networking

In Formula One racing, all cars must comply with a defined set of rules (the formula) and find ways to differentiate with a car, driver and support team that all follow the basic guidelines.

The same is true of automated trading systems. The formula is simple: each trading system consumes market data and produces orders. What happens in between is how each firm differentiates themselves. Just like in F1, anything that can be improved is up for evaluation when it comes to achieving fast, consistent performance.

Today we announced a partnership with Solarflare Communications to accelerate linux-based networking of high performance messaging. Let me take a minute to lay out what this partnership does for our customers.

It is well understood that in the software world trading performance is negatively impacted by “context switching”. This is an operating system issue, where the OS needs to manage state as control is passed from “kernel space” where the network drivers reside, to “user space” where the application runs. Each individual context switch is fast, but when you’re talking about hundreds of thousands of context switches per second the impact on CPU and latency adds up quickly.

The Solace Message Router doesn’t have this problem because there is no software or operating system in the execution path of processing market data, so overhead stays low and consistent even as rates increase. The applications that connect to Solace’s hardware run in software on operating systems, though, so publishers and receivers are still subject to the effects of context switching.

This is where Solarflare steps in. They provide a network interface card (NIC) called Solarstorm and a network driver called OpenOnload that use “kernel bypass” techniques to run the application and network driver together in user space and, well,   bypass the kernel. This allows the application side of the connection to process many more messages per second with lower and more consistent latency.

Most firms value consistent latency as much as they do raw speed, and that’s where the value of this partnership really shines. We tested a system featuring Solace and Solarflare running between 1 million and 5 million messages per second, and the average end-to-end latency ranged from 24 to 35 microseconds, while 99.9th percentile results ranged from 29 to 46. For those volumes, those are ridiculously tight latency bounds.

If  you’re a developer or architect who has fought with context switching for years kernel bypass may feel like cheating, but fortunately it’s completely within the rules.