The Solace connector for ingesting event data from Solace PubSub+ into Spark Streaming / Databricks is available as a free, community-supported connector.

Features

  • It supports processing of specific number of messages in a partition (BatchSize option can be used to configure the number of messages per partition)
  • Message are acknowledged to Solace once the commit method is triggered from Spark. Commit is triggered from Spark on successful write operation.
  • Message IDs are persisted in checkpoint location once commit is successful. This helps in deduplication of messages and guaranteed processing.
Endpoint Service
Apache Spark
Connector Type
Connector Endpoint

Still have questions?

Explore Other Connectors Get in Touch