Apache Spark

Features & Use Cases

• Batch Message Processing
  Ingest messages from Solace in micro-batches, enabling efficient integration with Spark Structured Streaming workloads.
• Reliable Acknowledgement Workflow
  Messages are acknowledged to the source broker only after Spark completes a successful commit, ensuring at-least-once delivery.
• Checkpointing and Deduplication
  Message IDs are persisted in Spark’s checkpoint location post-commit, supporting deduplication and exactly-once processing across restarts and failures.
• Supported Platforms
  Compatible with Spark runtimes, across both cloud-native and on-premise deployments.
• Message Publishing (Sink Support)
  Supports publishing messages from Spark back to Solace topics, enabling closed-loop processing and feedback-driven event pipelines.
• Dynamic Topic Publishing
  Messages can be routed to dynamically generated Solace topics based on record fields or transformation logic. This allows content-aware event routing and supports topic hierarchies for scalable message distribution.
• Secure Authentication Support
  The connector supports multiple authentication mechanisms for secure communication with Solace brokers:

• • Basic Authentication – Username and password.
  • Client Certificate Authentication (mTLS) – Uses client certificates for mutual TLS.
  • OAuth 2.0 (Client Credentials Flow) – Authenticates using access tokens obtained from an OAuth authorization server via the client credentials grant.