Asset Type:
Connectors
Provider:
Category:
Analytics & Stream Processing
Endpoint:
Target
Support:
Community
Apache Spark
The Solace Spark connector supports ingest of events from PubSub+ brokers using the Spark DataSource v2 API.
The connector features:
- Processing of messages in batches
- Message are acknowledged to the source broker once Spark triggers commit.
- Message IDs are persisted in checkpoint location once commit is successful. This helps in deduplication of messages and guaranteed processing.
- Support for various Spark version 3.x runtimes.
