Editor’s Note: The VMR (Virtual Message Router) mentioned in this post is now Solace PubSub+.
In part 1 of this series of blog posts, I introduced the concepts of using Solace for video streaming. In this second installment of the live video streaming over Solace series, let’s take a look at a simple streaming demo application called BroadcastMe. It’s designed to show how live video streaming can be implemented over Solace. The usage scenario is to multicast video from a single broadcaster to multiple stream viewers. The source code can be downloaded from Github.
Live video streaming is a little different from conventional video file playback. Stream video is usually transported using MPEG transport stream (MPEG-TS, MTS or TS). Very briefly, a transport stream specifies a container format which describes how to encapsulate packetized elementary streams with error correction and stream synchronization features. Within the transport stream, there are a number of sub-streams. For instance, a transport stream can contain a main data stream of MPEG codec (such as mpeg2, mp4, h264), as well as any number of non-MPEG codecs (such as ACS, DTS audio) or even text information like electronic program guide. The transport stream is then wrapped at the IP layer using UDP or RTSP (Real Time Streaming Protocol) for IP network delivery.
As the method of encoding and decoding transport streams for video/live stream playback has matured over the past decade, it does not make sense to break what already works by modifying codec to fit into the Solace messaging ecosystem. A more non-intrusive way to apply Solace’s data movement technology to live video streaming is to focus at the network IP layer. Specifically, I am referring to a streaming codec agnostic way of wrapping the payload of the transport layer protocol (such as UDP, RTSP) with Solace’s Solace Message Format (SMF). Here, the UDP payloads are the transport stream containers.
So, instead of delivering the transport stream container via the UDP protocol between a broadcaster and stream viewers, let’s intercept the UDP packet streams and re-inject its payload, encapsulated in SMF, into Solace for delivery. This way, Solace’s data movement features can be exploited to address various data/stream delivery problems like fan-out efficiency, geographical content routing and caching or even WAN optimization.
How it Works
BroadcastMe in its basic form is a proxy. It consists of two parts, an InputProxy and an OutputProxy. Typically, a live video stream will enter into Solace via InputProxy where it bridges the live stream source and Solace, and will exit Solace via OutputProxy where it bridges the stream viewer and Solace.
At the InputProxy, UDP data is read off from the socket and its content, i.e. the transport stream container, is then added as an attachment of an outgoing BytesXMLMessage (see snippets below). This is the encapsulation process.
// Read data off socket
dsocket.receive(packet);
data = new byte[packet.getLength()];
System.arraycopy(packet.getData(), packet.getOffset(), data, 0,
packet.getLength());
// Create SMF message and encapsulate
BytesXMLMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesXMLMessage.class);
msg.writeAttachment(data);
if (mode.equals("persistent")) {
msg.setDeliveryMode(DeliveryMode.PERSISTENT);
} else {
// Everything else, even if wrong spelling, just default to direct.
msg.setDeliveryMode(DeliveryMode.DIRECT);
}
// Send message
prod.send(msg, topic);
At the OutputProxy, the attachment part of the received BytesXMLMessage is read out and used as the message body of outgoing UDP packets (see snippets below). This is the decapsulation process.
// Pass msg content to UDP stream
DatagramPacket packet = new DatagramPacket(message.getAttachmentByteBuffer().array(),
message.getAttachmentContentLength(), address, port);
// Decapsulated and send to UDP
try {
dsocket.send(packet);
} catch (Exception ex) {
System.err.println("Encountered an Exception... "; + ex.getMessage());
}
So, what about the source of the live video stream? Are there any changes required? No, there are no modification required at the source side. The InputProxy picks up the video feed just like a normal video stream viewer device would, though transport protocols like UDP or RTSP. This design is intentional.
Likewise, what about at the destination, i.e. at the viewing device? You don’t need to change anything at the destination either. The OutputProxy delivers the video feed just like a normal video source device would through transport protocol like UDP or RTSP.
Let’s Get the Demo Running
To get BroadcastMe to showcase live video stream over Solace you will need:
- laptop with a video camera (for capturing live feeds)
- FFMpeg program (for encoding feeds)
- VLC program (for decoding feeds and viewing)
- BroadcastMe binary (get from Github and follow the included build instructions)
- JDK Environment 1.6 and above
- Solace Virtual Message Router (VMR) (download the VMR community edition from solace.dev)
Overview of Steps
- Prepare the Solace Virtual Message Router
- Link up the VMRs using Solace’s Multi-Node Routing (MNR) protocol
- Start the InputProxy
- Start live video stream capturing using FFMpeg
- Start the OutputProxy
- Start the VLC viewer
Detailed Procedure
- Prepare the Solace Message Router
- Install 2 VMRs (e.g. run VMR on Amazon AWS in 2 different availability zones). Refer to this link for installation procedures.
- Ensure the firewall is not blocking on port 55555, 55556 (default SMF ports)
- Ensure that message-vpn’s “Maximum Spool Usage” is configured (the OutputProxy uses temporary queue with topic subscription)
- Link up the VMRs using Solace’s Multi-Node Routing (MNR) protocol
The purpose is to show that the live video stream can, in fact, traverse through two different VMRs which can be geographically far apart.- Configure MNR between 2 VMRs. One with the host name ip-172-31-19-216 and the other ip-172-31-13-92.
ip-172-31-19-216# configure ip-172-31-19-216(configure)# routing ip-172-31-19-216(configure/routing)# create cspf neighbor ip-172-31-13-92 connect-via 52.56.121.135 ip-172-31-13-92# configure ip-172-31-13-92(configure)# routing ip-172-31-13-92(configure/routing)# create cspf neighbor ip-172-31-19-216 connect-via 52.221.59.15
- Confirm MNR is established. Run the ‘show cspf neighbor’ command and make sure that the state is showing ‘OK’.
ip-172-31-19-216# show cspf neighbor * Neighbor Ports State Cost Cost Uptime Data/Ctrl cfg act ---------------------------- ----------- -------- ---- ---- ---------------- ip-172-31-13-92 55555/—— Ok 100 100 0d 04h 41m 18s ip-172-31-13-92# show cspf neighbor * Neighbor Ports State Cost Cost Uptime Data/Ctrl cfg act ---------------------------- ----------- -------- ---- ---- ---------------- ip-172-31-19-216 55555/—— Ok 100 100 0d 04h 41m 26s
- Configure MNR between 2 VMRs. One with the host name ip-172-31-19-216 and the other ip-172-31-13-92.
- Start the InputProxy
- Download and compile BroadcastMe (see README.md for detail)
- Start the InputProxy from a shell environment of your choice. The command below instructs the InputProxy to connect to VMR with IP of 52.221.59.15 (i.e. the public IP address of host ip-172-31-19-216) using username default to message-vpn default, and to publish received video, from localhost port 1235, using direct messaging to topic channel/1.
$ ./inputProxy 52.221.59.15:55555 default default channel/1 direct 1235 verbose Input initializing... Jan 25, 2017 10:49:21 PM com.solacesystems.jcsmp.protocol.impl.TcpClientChannel call INFO: Connecting to host 'orig=52.221.59.15:55555, host=52.221.59.15, port=55555' (host 1 of 1, smfclient 2, attempt 1 of 1, this_host_attempt: 1 of 1) Jan 25, 2017 10:49:21 PM com.solacesystems.jcsmp.protocol.impl.TcpClientChannel call INFO: Connected to host 'orig=52.221.59.15:55555, host=52.221.59.15, port=55555' (smfclient 2) Connected. Control-C to exit
- Start live video stream capturing using FFMpeg
-
- Determine the laptop’s video capturing input device name with –list_devices option. For example, on a Macbook, the following indicates that “FaceTime HD Camera” is on “0”.
- Use ffmpeg to capture the video feed from a laptop camera and push the encoded output to an UDP port and address that matches the setting on InputProxy. In this demo, both processes are running on the same physical host.
./ffmpeg -f avfoundation -framerate 30 -video_size 640x480 -i "0" -f mpegts udp://localhost:1235
Under Linux environment, “video4linux2” should be passed in as the argument for -f and the input device (-i) normally defaults to /dev/video0.
-
- Start the OutputProxy
- Start the OutputProxy from a shell environment of your choice. The command below instructs the OutputProxy to connect to VMR with IP of 52.56.121.135 (i.e. the public IP address of host ip-172-31-13-92) using username default to message-vpn default, and to subscribe to topic channel/1 to receive messages and redirect them to IP address 127.0.0.1 at port 1239.
$ ./outputProxy 52.56.121.135:55555 default default channel/1 127.0.0.1 1239 verbose OutputProxy initializing... Jan 25, 2017 10:49:23 PM com.solacesystems.jcsmp.protocol.impl.TcpClientChannel call INFO: Connecting to host 'orig=52.56.121.135:55555, host=52.56.121.135, port=55555' (host 1 of 1, smfclient 2, attempt 1 of 1, this_host_attempt: 1 of 1) Jan 25, 2017 10:49:24 PM com.solacesystems.jcsmp.protocol.impl.TcpClientChannel call INFO: Connected to host 'orig=52.56.121.135:55555, host=52.56.121.135, port=55555' (smfclient 2) Connected. Control-C to exit
- Confirm that topic subscriptions (e.g. channel/1 etc.) are propagated in the MNR network with the ‘show smrp subscriptions’ CLI command.
ip-172-31-19-216# show smrp subscriptions Flags Legend: T - Destination Type (C=local-client, Q=local-queue R=remote-router) P - Subscription Persistence (P=persistent, N=non-persistent) R - Redundancy Type for Local Destinations (P=primary, B=backup S=static -=not-applicable) Message VPN : default (exported: Yes; 100% complete) Destination Name Flags BlkID DTO Subscription T P R Prio ------------------------ - - - ----- ---- ------------------------------------ #client C N S 0 P1 #MCAST/> #client C N S 0 P1 #SEMP/ip-172-31-19-216/> #client C N S 0 P1 #P2P/ip-172-31-19-216/#client/> #client C N S 0 P1 #P2P/v:ip-172-31-19-216/#client/> #client C N S 0 P1 #SEMP/v:ip-172-31-19-216/> ip-172-31-13-92 R N - 0 DA #MCAST/> ip-172-31-13-92 R N - 0 DA #SEMP/ip-172-31-13-92/> ip-172-31-13-92 R N - 0 DA #P2P/ip-172-31-13-92/#client/> ip-172-31-13-92 R N - 0 DA #P2P/v:ip-172-31-13-92/#client/> ip-172-31-13-92 R N - 0 DA #SEMP/v:ip-172-31-13-92/> v:ip-172-31-13-92 R N - 21 DA channel/1 ip-172-31-13-92# show smrp subscriptions Flags Legend: T - Destination Type (C=local-client, Q=local-queue R=remote-router) P - Subscription Persistence (P=persistent, N=non-persistent) R - Redundancy Type for Local Destinations (P=primary, B=backup S=static -=not-applicable) Message VPN : default (exported: Yes; 100% complete) Destination Name Flags BlkID DTO Subscription T P R Prio ------------------------ - - - ----- ---- ------------------------------------ #client C N S 0 P1 #MCAST/> #client C N S 0 P1 #SEMP/ip-172-31-13-92/> #client C N S 0 P1 #P2P/ip-172-31-13-92/#client/> #client C N S 0 P1 #P2P/v:ip-172-31-13-92/#client/> #client C N S 0 P1 #SEMP/v:ip-172-31-13-92/> #P2P/QTMP/v:ip-172-31-13 Q P P 21 DA channel/1 -92/29f6f3a8-08be-4014 -8e72-e205af594849 ip-172-31-19-216 R N - 0 DA #MCAST/> ip-172-31-19-216 R N - 0 DA #SEMP/ip-172-31-19-216/> ip-172-31-19-216 R N - 0 DA #P2P/ip-172-31-19-216/#client/> ip-172-31-19-216 R N - 0 DA #P2P/v:ip-172-31-19-216/#client/> ip-172-31-19-216 R N - 0 DA #SEMP/v:ip-172-31-19-216/>
The OutputProxy create a temporary queue (#P2P/QTMP/v:ip-172-31-13-92/29f6f3a8-08be-4014-8e72-e205af594849) and maps topic subscription “channel/1” to attract the messages.
- Confirm that video streams are flowing between the VMRs by checking the Total Message/Bytes Sent and Received counter in neighbor statics.
ip-172-31-19-216# show cspf neighbor ip-172-31-13-92 stats Neighbor : ip-172-31-13-92 Connect Via : 52.56.121.135:55003 Control Port : unspecified Received Sent -------------------- -------------------- Total Messages 3399 6118 Control Messages 2486 2482 Data Messages 913 3636 Total Bytes 1380320 4010413 Control Bytes 461506 462290 Data Bytes 918814 3548123 Ingress (msg/sec) Egress (msg/sec) -------------------- -------------------- Current Rate (1 sec sample) 1 48 Average Rate (60 sec interval) 0 9 Ingress (byte/sec) Egress (byte/sec) -------------------- -------------------- Current Rate (1 sec sample) 181 44353 Average Rate (60 sec interval) 28 9824 Total Ingress Discards 0 Total Egress Discards 0 ip-172-31-13-92(admin)# show cspf neighbor ip-172-31-19-216 stats Neighbor : ip-172-31-19-216 Connect Via : 52.221.59.15:55003 Control Port : unspecified Received Sent -------------------- -------------------- Total Messages 7808 3396 Control Messages 2478 2483 Data Messages 5330 913 Total Bytes 5669019 1379594 Control Bytes 461491 460780 Data Bytes 5207528 918814 Ingress (msg/sec) Egress (msg/sec) -------------------- -------------------- Current Rate (1 sec sample) 43 1 Average Rate (60 sec interval) 26 0 Ingress (byte/sec) Egress (byte/sec) -------------------- -------------------- Current Rate (1 sec sample) 35315 181 Average Rate (60 sec interval) 25662 42 Total Ingress Discards 0 Total Egress Discards 0
- Start the OutputProxy from a shell environment of your choice. The command below instructs the OutputProxy to connect to VMR with IP of 52.56.121.135 (i.e. the public IP address of host ip-172-31-13-92) using username default to message-vpn default, and to subscribe to topic channel/1 to receive messages and redirect them to IP address 127.0.0.1 at port 1239.
- Start the VLC viewer
- Start VLC and choose Media/Open Network Stream (usually under File -> Open Network)
- Enter, in the address field, the URL matching the redirected address and port specified by the OutputProxy, e.g. udp://@127.0.0.1:1239. Don’t forget the “@” symbol after the “udp://” and before the address.
Subscribe to Our Blog
Get the latest trends, solutions, and insights into the event-driven future every week.