What is an Event Streaming Platform?

I’ve long struggled with the term “event streaming”. It seems like a simple enough term but I’ve found it often means different things to different people. To some, including myself, event streaming simply means streaming events in real-time (or near real-time for you purists) from one or more sources to one or more destinations. So what is event streaming and what is an event-streaming platform?

Simply put, an ‘event’ is data. But more specifically, ‘event’ tends to represent a change in state of data, like a sensor signaling a change in temperature, a field changing in a database, a bank deposit being completed, a checkout button being clicked in an e-commerce app, etc. These can be system level events or business events. All are valid events.

To some people, the term event streaming can take on a the broader meaning of event stream processing, which is when event/data streams are ingested and calculations performed and data is transformed/enriched to create new insights or higher level events and possibly take an appropriate action. Lastly, I sometimes see event streaming used in place of the narrower use case of streaming analytics – the analysis of a window of events/data (usually bounded by time) to detect patterns or correlations between the data points in an effort to aid in decision making.

But enough about what I personally think, let’s look to how the industry analysts view event streaming.

Gartner and Forrester’s Take on Event Streaming

Gartner more closely aligns event streaming with event stream processing and streaming analytics, although they estimate that roughly 45% of event stream processing (ESP) loads are basic data movement and processing, rather than analytics.¹ Interestingly, Gartner also notes one of the more common event stream processing use case as stream data integration – effectively performing real-time ETL and data integration.

But I really like how a recent Forrester report defines and categorizes streaming – albeit the focus is more on data vs events but as we’ve discussed – they are the same.

In the report Now Tech: Stream Data Platforms, Q2 2021, Forrester analyst Mike Gualtieri defines a streaming data platform as:

Enterprise software that connects to multiple, disparate live data sources (streaming data) to expose and/or deliver to subscribing applications, analyze to gain real-time insights, and/or recombine for continuous integration to downstream data repositories or applications.”²

Forrester further segments this market into Streaming Data Flow platforms, Streaming Data Analytics platforms, and Streaming Data Processing platforms. These seem self-explanatory and more closely mimic and delineate the various definitions outlined at the beginning of this blog post.

What is an Event Streaming Platform?

By extension, the software platforms that support these various capabilities would all be an event streaming platform. Some event streaming platforms might specialize in highly reliable and resilient streaming data flow, while others will specialize in streaming analytics or stream processing. Forrester’s report does exactly this – identifies the streaming platforms by their specialties and categorizes them by size (large, midsize, and small). Solace is identified as a large vendor specializing in streaming data flow capability, which seems spot on.

While the data flow, analytics, and processing capabilities are very important in these platforms, what Forrester’s report does not touch on is the follow-on requirement of management of the event streams. It is one thing to move and process all these event or data streams, but who manages the streams themselves? The definition, sharing, governance, and deprecation of the streams? This is where an event streaming and management platform comes in – like Solace PubSub+ where the event brokers handle the streams themselves and PubSub+ Event Portal looks after the management aspect.

Comparing Event Streaming Platforms and Technologies for Event-Driven ArchitectureVP of Product Marketing, Chris Wolski, compares event streaming platforms and other EDA tech and explains why he thinks PubSub+ is leading the pack.Read Blog Post

What about Event-Driven Architecture?

While we have outlined the various definitions and segmentations of event and data streaming, it is very important not to equate these exclusively with event-driven architecture (EDA). EDA certainly utilizes event streaming technology concepts (ie. publish-subscribe) and components (ie. event brokers), but it is equally important to recognize that EDA is an architectural design pattern for application building centered around decoupled operations responding to one or more event notifications in a real-time and asynchronous manner.

You can utilize event streaming technologies, perform event stream processing and streaming analytics but still not truly be doing event-driven architecture. More often than not, EDA is implemented for the purpose of choreographing real-time operational applications – think event-driven microservices as one example.

We have also seen a trend recently whereby customers are using EDA concepts and technology for event-driven integration, similar to what Gartner identifies as stream data integration. In other words, connecting disparate systems together via event flows and trigger updates to system B based on real-time changes to data in system A, regardless of where these systems live, in public or private clouds, or on-premises.

The take-away here is there are many ways to look at event streaming. Event management and many other technologies can address your needs, whatever they may be. It is most important to understand what kind of event streaming you need for your use case, what kind of management tooling you need (if any), and selecting the right technology for the job.

_{1 Source: Gartner “Market Guide for Event Stream Processing”, 14 Dec 2020, Roy Schulte, Pieter den Hamer
2 Source: Forrester “Now Tech: Streaming Data Platforms, Q2 2021”, 7 April 2021, Mike Gualtieri}