Snowflake

The PubSub+ Connector for Snowflake provides integration of source events from a PubSub+ event broker to a Snowflake database via the Snowflake Snowpipe Streaming API (default) or the Snowpipe API using stages. The connector provides for high performance ingest of event mesh data to Snowflake. This connector does not provide Snowflake -> PubSub+ data flows.

This integration is configured via “workflows” in your connector instance. A workflow is a source-to-processing-to-target data pipeline configured within the connector runtime. Each workflow (you can have up to 20 defined per connector instance), defines a source (e.g. PubSub+ queue), any header processing necessary for the target, and a target destination. When active, events will stream across these Workflows.

The connector is available as:

• A runnable package based on a Java JAR file including a start script
• A container image suitable for running in a container runtime such as Docker or Podman

The PubSub+ Connector  is a “self-contained connector” from Solace. All self-contained connectors share a common architecture and provide a number of enterprise services to the connectors such as:

• A local management server accessible over HTTP(s) and JMX exposing endpoints for:
  • Health check
  • Metrics monitoring
  • Log file access
  • Workflow adminstration (start & stop workflows)
• A common set of configuration options for:
  • logging – log levels, log file size, archive and rollover rules, appenders to export to other log services
  • security setup for management endpoints – authentication and authorization to the endpoints, TLS for HTTPS endpoints
• Various runtime deployment options:
  • Standalone
  • Active_Standby – for redundancy (you can have more than 1 standby instance)
  • Active_Active – for horizontal scaling (where the source of data will support multiple active consumers such as a non-exclusive queue in PubSub+