Hardware trading: The machines are taking over…

With each day that passes it becomes more obvious that hardware is replacing software in the critical path of high performance trading. It doesn’t take a genius to see what’s happening, as the evidence is everywhere:

• Feed handlers: All of the recent entrants into the feed handler market are using custom hardware.
• Messaging middleware: Appliances from companies like Solace are winning a larger footprint for market data delivery and back-office order routing.
• Algo engines: Leading algo trading engines are heavily leveraging FPGAs and GPUs to do ultra-fast simulations that accelerate trading decisions.
• TCP offload engines: TOEs are taking the place of software TCP stacks on servers.
• Trading apps:  We’ve even seen a flurry of requests from high performance trading firms looking to execute pre-trade risk checking directly in network processors inline with the hardware messaging bus.

Feed handlers, messaging, algos, network stacks, risk-checking all in hardware. Hmmm…that’s pretty much everything. Even state-of-the-art latency monitoring tools are now implemented in hardware to keep up with all of the other hardware. There’s no question the tide has turned. About three years ago, you needed a compelling reason to take a chance on hardware acceleration. Today you have to have a pretty compelling reason to stick with software.

Hardware’s advantage is about more than just speed. The unappreciated value of appliances is how much easier they are to install, configure and operate. They offer a self-contained, internally consistent operating environment: plug them in and you’re off, full-steam ahead. Think about your cable box or WiFi router at home — are they easier or harder to deal with than Windows software on your PC which requires 3 “critical” security patches a week?

Another argument for software is that because of Moore’s Law, CPUs will eventually catch up with special-purpose hardware. But Moore’s Law applies equally to CPUs, FPGAs, network processors and all kinds of chipsets. They all expand their transistor density at about the same rate. The difference is that CPUs are “clocked out” at about 3 GHz. Single threaded performance is not increasing any more and all the extra transistors Moore’s Law provides to CPUs are going towards additional cores and cache. If you’ve got a problem that can be parallelized, and the programming skill to do so, you can get more performance, but at the cost of more complexity.

Targeted functionality implemented in specialized hardware such as FPGAs, GPUs and network processors is already outperforming software on CPUs by a ten-to-one margin, and most of these chips are executing at clock speeds less than 1 GHz. These chips have lots of runway to double in raw speed many more times before they run into the execution ceiling problems CPUs have now. Anyone looking at this objectively would have to conclude that specialized hardware is positioned to open a wider performance gap on software as we go forward.

Here’s another way to think about the question: Once IP routers moved into hardware, did CPU speed improvements cause people to go back to IP routing in software? What about web security appliances, were they a blip that went away as servers got faster? Did people stop using graphics chips for video games as general CPUs got smarter?

The history of computing has shown that when specialized hardware proves its value, it sticks around (for good). Hardware is rapidly reshaping trading and it’s not a trend that’s likely to reverse any time soon.