PubSub+ Connector for Files

The PubSub+ Connector for Files provides integration of file-based data, such as on filesystems or in an (s)FTP server with PubSub+ Event Brokers and the event mesh.

The connector works by “event-enabling” files by turning file data into Event messages for either ingest into a PubSub+ broker or from a broker to a file. Using the event mesh, files can participate in message-based application patterns like microservices and event-driven architectures.

This 2-way integration is configured via “workflows” in your connector instance. A workflow is a source-to-processing-to-target data pipeline configured within the connector runtime. Each workflow (you can have up to 20 defined per connector instance), defines a source (e.g. a file on an SFTP server), any header processing necessary for the target, and a target destination (e.g. a PubSub+ Topic). When active, events will stream across these Workflows.

When doing file replication, the workflow is configured to be a source file -> processing -> Solace queue/topic -> target file where the intermediate Solace destination is used to transport the file from source to its replication target.

The connector is available (below) as:

• A runnable package based on a Java JAR file including a start script
• A container image suitable for running in a container runtime such as Docker or Podman

The PubSub+ Connector for Files is a “self-contained connector” from Solace. All self-contained connectors share a common architecture and provide a number of enterprise services to the connectors such as:

• A local management server accessible over HTTP(s) and JMX exposing endpoints for:
  • Health check
  • Metrics monitoring
  • Log file access
  • Workflow adminstration (start & stop workflows)
• A common set of configuration options for:
  • logging – log levels, log file size, archive and rollover rules, appenders to export to other log services
  • security setup for management endpoints – authentication and authorization to the endpoints, TLS for HTTPS endpoints
• Various runtime deployment options:
  • Standalone
  • Active_Standby – for redundancy (you can have more than 1 standby instance)
  • Active_Active – for horizontal scaling (where the source of data will support multiple active consumers such as a non-exclusive queue in PubSub+)

Note: We have temporarily taken down the download links for this connector as a new version is imminent and will be a better starting point for your file integration needs.