RTSP Sink

From RidgeRun Developer Wiki

Template:Eval SDK Download, Demo Image download and Contact Us buttons

Overview

RTSP Sink is a GStreamer element which permits high performance streaming to multiple computers using the RTSP / RTP protocols. The rtspsink element leverages previous logic from RidgeRun's RTSP server with extensions to create a GStreamer sink element providing benefits like greater flexibility to application integrate and easy gst-launch based testing.

With RTSP Sink you have the flexibility to stream different content to the same client, such as streaming audio and video to a client. You also can send different streams to different clients. This means that within a single GStreamer pipeline you can stream multiple videos, multiple audios and multiple lip-sync audio+video streams, each one to a different client using a different RTSP mapping. In the examples section different streaming possibilities are shown.

Features

The rtspsink GStreamer element supports the following features:

  • Available for GStreamer-1.0 and Gstreamer-0.10 versions
  • Configurable mappings for each stream. The mapping is the text string used at the end of the RTSP URL: rtsp://$SERVER_IP/$MAPPING
  • Configurable TCP port number
  • Automatic payloader detection according to the negotiated pipeline capabilities
  • Multiple independent video-only streams
  • Multiple independent audio-only streams
  • Multiple independent audio+video streams
  • Support for:
    • H264 video encoding
    • MPEG4 video encoding
    • VP8 video encoding
    • MJPEG video encoding
    • TRANSPORT STREAM A/V
    • AAC audio encoding
    • AC3 audio encoding
    • PCMA audio encoding
    • PCMU audio encoding
    • OPUS audio encoding
  • Designed for use in embedded products; supports zero-copy pipelines
  • Multicast networking support
  • Control each stream independently by setting properties
  • RTSP over HTTP Tunneling support
  • Authentication stream control by setting user and password

The number of simultaneous streams is limited by the hardware capabilities.

Getting the code

RTSP Sink is an add-on to RidgeRun's professional SDK. You can purchase RTSP Sink, with full source code, from the RidgeRun Store or using the Shopping Cart mentioned near to Table of Contents above.

RidgeRun also makes a binary-only evaluation version available.

Evaluating RTSP Sink

To help you test RTSP Sink, RidgeRun can provide an evaluation version of the plug-in. The following table summarizes the features available in both the professional and evaluation version of the element.

Feature Professional Evaluation
Video Streaming Y Y
Audio Streaming Y Y
A/V Streaming Y Y
Multiple Parallel Streams Y Y
Multiple Encoding Formats Y Y
Multicast Support Y Y
Independent Stream Control Y Y
Authorization Stream Control Y Y
Unlimited Streaming Time Y N
Source Code Y N

The evaluation version will limit the streaming time of each pad to 5 minutes (at 30fps).

To use the binary provided by RidgeRun, append the following environment variable:

GST_PLUGIN_PATH=$PATH_TO_BINARY_DIRECTORY gst-inspect-1.0 rtspsink
GST_PLUGIN_PATH=$PATH_TO_BINARY_DIRECTORY gst-launch-1.0 ...

Additionally, you may just copy the binary into the standard GStreamer plug-in search path. Refer to the examples section for possible combinations and use cases.

Building the plug-in

Although rtspsink source code is delivered ready to be built along with RidgeRun's SDK, the code is platform independent and can be built on a host PC or other build environment that is configured to build GStreamer plug-ins.

Building rtspsink using RidgeRun SDK

The source code for RTSP sink can be found in

$DEVDIR/proprietary/gst-rtsp-sink

The element will be built along with the SDK if selected on the configuration menu.

make config
 '-> Proprietary software
       '-> [*] RidgeRun RTSP GStreamer sink element
             '-> [] Enable Multicast support


The RTSP Sink can be built manually by:

cd $DEVDIR/proprietary/gst-rtsp-sink
make
make install

Notes:

  • Building the gst-rtsp-sink as a standalone may fail if the package dependencies are not first built properly.
  • Enable the Enable Multicast support option to build the element with multicast support.

Building for a host PC

  • Build the code
cd $DEVDIR/proprietary/rr_rtsp_sink/src # Note the src subdirectory
./autogen.sh
./configure --libdir=/usr/lib/x86_64-linux-gnu/     # GStreamer will look for its plug-ins in this standard location
make
  • Install the plugin
sudo make install                       # You will need root privileges to install the plugin

Note: The default port used by RTSP is 554. Running RTSP Sink on a host PC may require root privileges to open the appropriate sockets (port numbers less than 1024). If the debug reveals a permission problem, then run the pipeline using sudo. You should be able to run GStreamer pipelines with rtspsink if the specified port number is 1024 or greater.

Basic usage

RTSP Sink is a GStreamer element which supports consuming the stream from multiple pipeline branches, each one describing a different media to stream.

Requesting pads

The RTSP Sink pads are requested using regular GStreamer notation. For example, using GStreamer textual pipeline notation, attaching three different h.264 encoded video branches to the sink the pipeline looks like the following:

gst-launch rtspsink name=sink \
< branch 1 > ! sink. \
< branch 2 > ! sink. \
< branch 3 > ! sink. 

Programatically, the pads would be requested as the following:

rtspsinkpad1 = gst_element_get_request_pad(rtspsink, "sink_%d");
gst_pad_link (branch1pad, rtspsinkpad1);
 
rtspsinkpad2 = gst_element_get_request_pad(rtspsink, "sink_%d");
gst_pad_link (branch2pad, rtspsinkpad2);

rtspsinkpad3 = gst_element_get_request_pad(rtspsink, "sink_%d");
gst_pad_link (branch3pad, rtspsinkpad3);

Graphically, the pipeline would look something like:

Since we didn't specifying the mapping, which is described below, all three pipelines would be mapped to /test.

Combining and naming streams

Each stream needs to be mapped to a different URL so the different clients can access them independently. This is called the mapping of the stream. The RTSP Sink element will try to read the mapping from the caps it negotiates with the upstream element. This means that assigning a mapping to a stream is as simple as setting it in the caps, using the mapping= value. The only rule is that the mapping must always start with a leading slash "/". If no mapping is explicitly provided the it will default to /test.

Each mapping will be treated as an individual stream. This means that if you want to combine an audio and a video branch within a single A/V stream, it is as simple as using the same mapping name. The following pipeline will produce 3 independent streams: a video mapped to "/video", an audio mapped to "/audio" and an audio+video mapped to "/audiovideo".

gst-launch rtspsink name=sink \
< branch 1  > ! capsfilter caps="video/x-h264, mapping=/video" ! sink. \
< branch 2  > ! capsfilter caps="audio/mpeg, mapping=/audio" ! sink. \
< branch 3v > ! capsfilter caps="video/x-h264, mapping=/audiovideo" ! sink. \
< branch 3a > ! capsfilter caps="audio/mpeg, mapping=/audiovideo" ! sink.

This will combine branches 3v and 3a as a single streaming under the name of "/audiovideo". This can be shown graphically on figure 2.

Supported formats

The following table conveniently summarizes the supported encoding formats and their respective mimetypes for you to build as

gst-launch ... ! mimetype, mapping="/test" ! rtspsink


Supported formats and their mimetypes
Format Mimetype
H264 video video/x-h264
VP8 video video/x-vp8
MPEG4 video video/mpeg video/x-jpeg
MPEG TS video/mpegts
MJPEG video image/jpeg
AAC audio audio/mpeg
AC3 audio audio/x-ac3 audio/ac3
PCMU audio audio/x-mulaw
PCMA audio audio/x-alaw
OPUS audio audio/x-opus

Service configuration

RTSP is high level protocol that negotiates streaming internals with every client that intends to connect to a mapping. In order to request a stream, the client must connect to the server at a specific port where the RTSP server is up and waiting for connections. The port at which the server is listening to, and the clients must talk to is called the service and it is typically set to TCP port 554. RTSP Sink allows to configure the service by means of a GStreamer property as follows:

gst-launch audiotestsrc ! ffenc_aac ! capsfilter caps="audio/mpeg, mapping=/mystream" ! rtspsink service=3000

If no service is specified then it default to 554. Since all ports below 1024 require root priority to use, port 554 could cause the pipeline to fail. Specify a port number 1024 or above, i.e. 3000, to run the pipeline at normal user priority. There are other methods that are possible to avoid the need to have root privileges.

Multicast

RTSP sink has the ability to send data from a single server to multiple clients without knowing who will receive the data via multicast. To enable multicast mode, the property must be set to true, if no multicast is specified then it default to false

Multicast has some properties:

Multicast properties
Properties set name Default value
Multicast multicast false
Minimum address multicast-ip-min 254.2.0.1
Maximum address multicast-ip-max 254.2.0.10
Minimum port multicast-port-min 5000
Maximum port multicast-port-max 5010
TTL multicast-ttl 16

Minimum and maximum address is the address range that multicast can take to connect a single server. If the first address is already in use, it will connect to the next address, that is not in use. Port range has similar behaviour, it has minimum and maximum port, and connects to the first port in the range that is not in use.

Time-to-live 'ttl' property in only available when multicast is set to true, it can takes values in the range of 0 to 255.

Independent Stream Control

RTSP Sink allows the user to control independent streams by setting properties on them. internally each stream is consumed by an appsink, the user can access the appsink properties as follows:

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264, mapping=/stream1  ! rtspsink service=5000 appsink0::sync=false

if sync appsink property is set on false, CPU usage is 320%, but if sync=true, CPU usage is 9,3%.

you can set properties for one appsink or more as follows:

gst-launch-1.0 rtspsink service=5000 name=sink appsink0::sync=true appsink1::max-buffers=1000 \
videotestsrc ! vp8enc ! capsfilter caps="video/x-vp8, mapping=/stream0" ! sink. \
audiotestsrc ! avenc_ac3 ! capsfilter caps="audio/x-ac3, mapping=stream1" ! sink.

the full list of appsink that you can configure is detailed as follows:

Appsink properties
Property Function Default value
sync Sync on the clock true
max-lateness Maximum number of nanoseconds that a buffer can be late before it is dropped -1
qos Generate Quality-of-Service events upstream false
async Go asynchronously to PAUSED true
ts-offset Timestamp offset in nanoseconds 0
enable-last-sample Enable the last-sample property true
blocksize Size in bytes to pull per buffer 4096
render-delay Additional render delay of the sink in nanoseconds 0
throttle-time The time to keep between rendered buffers 0
max-bitrate The maximum bits per second to render 0
emit-signals Emit new-preroll and new-sample signals false
max-buffers The maximum number of buffers to queue internally 0
drop Drop old buffers when the buffer queue is filled false
wait-on-eos Wait for all buffers to be processed after receiving an EOS true

RTSP over HTTP Tunneling

HTTP tunneling is used to encapsulated RTSP communications using the HTTP protocol, permitting to be use the application in restricted conditions of the network connectivity, such as firewalled networks and networks behind proxy servers. The client should set the tunneled option by sending a GET request, and sending a tunneling signal to the server.

VLC

VLC has the ability to encapsulate RTSP over HTTP tunneling. First you have to configure the HTTP connection and HTTP tunnel port number as follow:

 -> Tools
    -> Preferences
       -> All
          -> Input / Codecs
             -> Demuxers   
                -> RTP/RTSP

Here enable the "Tunnel RTP and RTSP over HTTP" option and change the port used for RTSPsink stream.

Server

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264, mapping=/stream1  ! rtspsink service=5000

Client

SERVER_IP=localhost
PORT=5000
MAPPING=stream1
vlc rtsp://$SERVER_IP:$PORT/$MAPPING

In this case HTTP Tunnel port is 5000.

RTSPsrc

RSTPsrc has a rtsph URI protocol to set a HTTP tunneling. The following pipelines shows how works:

Server

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264, mapping=/stream1  ! rtspsink service=5000

Client

SERVER_IP=localhost
PORT=5000
MAPPING=stream1
gst-launch-1.0 rtspsrc location="rtsph://$SERVER_IP:$PORT/$MAPPING" ! rtph264depay ! \
avdec_h264 ! queue ! xvimagesink sync=false

Authentication Stream Control

RTSPsink check if the current user is allowed to perform requested actions, the user and password will be set through "auth" property.

Auth property has next format:

auth=user1:password1,user2:password2,....userN:passwordN

Example:

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264, mapping=/stream1  ! rtspsink service=5000 auth=Adrian:Cervantes,user:pass 


Considerations:

  • If "auth" property is not set, the client can connect to the server without any user and password.
  • If some user entry is malformed, RTSPsink will ignore that user and set the rest.
rtspsink service=5000 auth=Adrian:Cervantes,user1,user:pass 

In that case RTSPsink will ignore user1.

  • If a user has an empty password, the client will not need a password to connect.
rtspsink service=5000 auth=user: 


Client can be connected through VLC and rtspsrc as follow:

SERVER_IP=localhost
PORT=5000
MAPPING=stream1 

Using VLC:

vlc rtsp://user:password@$SERVER_IP:$PORT/$MAPPING

another way, VLC will ask you the user and password:

vlc rtsp://$SERVER_IP:$PORT/$MAPPING

Using RTSPsrc:

gst-launch-1.0 rtspsrc location="rtsp://user:password@$SERVER_IP:$PORT/$MAPPING" ! rtph264depay ! \
avdec_h264 ! queue ! xvimagesink sync=false

GStreamer-1.0 examples

The following section provides various RTSP Sink use cases. For all of them, the different streams can be viewed on a host PC using VLC as follows:

SERVER_IP=localhost
PORT=5000
MAPPING=stream1

vlc rtsp://$SERVER_IP:$PORT/$MAPPING

where $SERVER_IP is the IP address of the RTSP server, $MAPPING is the name of the mapping that the client wants to connect to, and $PORT is the service at which the server is listening to.


Single MPEG4 video streaming

gst-launch-1.0 videotestsrc ! avenc_mpeg4 ! video/mpeg, mapping=/stream1  ! rtspsink service=5000

Single H264 video streaming

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264, mapping=/stream1  ! rtspsink service=5000

Single VP8 video streaming

gst-launch-1.0 videotestsrc ! vp8enc ! video/x-vp8, mapping=/stream1  ! rtspsink service=5000

Single JPEG image streaming

gst-launch-1.0 videotestsrc ! jpegenc ! image/jpeg, mapping=/stream1  ! rtspsink service=5000

Single AAC audio streaming

gst-launch-1.0 audiotestsrc ! faac ! audio/mpeg, mapping=/stream1  ! rtspsink service=5000

Single AC3 audio streaming

gst-launch-1.0 audiotestsrc ! avenc_ac3 ! audio/x-ac3, mapping=stream1 ! rtspsink service=5000

Single PCMU audio streaming

gst-launch-1.0 audiotestsrc ! mulawenc ! audio/x-mulaw, mapping=stream1 ! rtspsink service=5000

Single PCMA audio streaming

gst-launch-1.0 audiotestsrc ! alawenc ! audio/x-alaw, mapping=stream1 ! rtspsink service=5000 

Single OPUS audio streaming

Server

gst-launch-1.0 audiotestsrc ! opusenc ! audio/x-opus, mapping=stream1 ! rtspsink service=5000 

Client

SERVER_IP=localhost
PORT=5000
MAPPING=stream
gst-launch-1.0 rtspsrc location="rtsp://$SERVER_IP:$PORT/$MAPPING" ! rtpopusdepay  ! opusdec ! alsasink

Video test RTSPsink multicast

gst-launch-1.0 videotestsrc ! x264enc ! video/x-h264, mapping=/stream1  ! rtspsink \
service=5000 multicast=true multicast-ip-min=239.255.255.253 multicast-ip-max=239.255.255.255 \
multicast-port-min=5000 multicast-port-max=6000  multicast-ttl=128

Advanced examples

Dual H264 video streaming

gst-launch rtspsink name=sink \
v4l2src ! x264enc ! capsfilter caps="video/x-h264, mapping=/stream1" ! sink.
videotestsrc ! x264enc ! capsfilter caps="video/x-h264, mapping=/stream2" ! sink.

Audio+Video Streaming

gst-launch rtspsink name=sink \
v4l2src ! x264enc ! capsfilter caps="video/x-h264, mapping=/stream" ! sink.
alsasrc ! ffenc_aac ! capsfilter caps="audio/mpeg, mapping=/stream" ! sink.

Audio+Video streaming plus H264 single streaming

gst-launch rtspsink name=sink \
v4l2src ! x264enc ! capsfilter caps="video/x-h264, mapping=/stream1" ! sink.
alsasrc ! ffenc_aac ! capsfilter caps="audio/mpeg, mapping=/stream1" ! sink.
videotestsrc ! x264enc ! capsfilter caps="video/x-h264, mapping=stream2" ! sink.

Transport Stream payloader

Often RTSP is used with the RTP payloader encoding. It is also possible to use RTSP with the Transport Stream (TS) payloader.

PC and embedded target hardware example

This example pipelines was tested on a Leopardboard 368 with an Aptina mt9p031 sensor.

Pipeline to run on target hardware

Generate streaming:

RAW_CAPS='video/x-raw-yuv, width=1920, height=1088, framerate=(fraction)30/1'
ENCODER_SETTINGS='encodingpreset=2 ratecontrol=2 targetbitrate=2000000 intraframeinterval=30 idrinterval=90 bytestream=true'

gst-launch v4l2src input-src=camera always-copy=false ! capsfilter caps=$RAW_CAPS ! dmaiaccel ! dmaiperf ! /
           dmaienc_h264 $ENCODER_SETTINGS ! queue ! mpegtsmux ! capsfilter caps="video/mpegts,mapping=/stream" ! rtspsink service=5000
Pipeline to run on host PC

View streaming video:

TARGET_IP_ADDRESS=192.168.0.10 # replace with the IP address of your target hardware
PORT=5000 # same as service property value in rtspsink caps

gst-launch rtspsrc location="rtsp://$TARGET_IP_ADDRESS:$PORT/stream" ! rtpmp2tdepay ! mpegtsdemux ! queue ! \
           ffdec_h264 ! xvimagesink sync=false
Using VLC or mplayer to view streaming video
TARGET_IP_ADDRESS=192.168.0.10 # replace with the IP address of your target hardware

vlc rtsp://$TARGET_IP_ADDRESS:$PORT/stream

mplayer rtsp://$TARGET_IP_ADDRESS:$PORT/stream

HOST PC example

This example pipelines are intended to be run on the PC.

Pipelines to run on Host PC

Generate streaming:

gst-launch-1.0 videotestsrc ! 'video/x-raw, format=NV12, width=1920, height=1088, framerate=(fraction)30/1' ! x264enc ! \
h264parse ! queue ! mpegtsmux ! capsfilter caps="video/mpegts,mapping=/stream" ! rtspsink service=5000

View streaming video:

TARGET_IP_ADDRESS=192.168.0.10 # replace with the IP address of your PC
PORT=5000 # same as service property value in rtspsink caps

gst-launch-1.0 rtspsrc location="rtsp://$TARGET_IP_ADDRESS:$PORT/stream" ! rtpmp2tdepay ! tsparse ! tsdemux ! queue ! h264parse ! \
avdec_h264 ! autovideosink sync=false
Using VLC or mplayer to view streaming video
TARGET_IP_ADDRESS=192.168.0.10 # replace with the IP address of your PC
PORT=5000 # same as service property value in rtspsink caps

vlc rtsp://$TARGET_IP_ADDRESS:$PORT/stream

mplayer rtsp://$TARGET_IP_ADDRESS:$PORT/stream

Performance

iMX6

RTSP sink was tested on iMX6 Nitrogen board, with v5640 MIPI camera. Using perf you can get CPU load, BPS and FPS, as shown below:

Pipeline

 gst-launch-1.0 -e imxfslv4l2src device=/dev/video1 ! videoconvert ! queue ! vpuenc ! "video/x-h264, mapping=/stream1" ! \
 perf print-arm-load=true ! rtspsink service=5000 multicast=true

FPS and CPU load

Timestamp: 0:06:06.671329376; Bps: 14060; fps: 25.4; CPU: 1; 
Timestamp: 0:06:07.706948043; Bps: 14494; fps: 25.12; CPU: 2; 
Timestamp: 0:06:08.744248709; Bps: 94080; fps: 25.7; CPU: 1; 
Timestamp: 0:06:09.781531376; Bps: 97170; fps: 25.7; CPU: 0; 
Timestamp: 0:06:10.818393710; Bps: 78921; fps: 25.9; CPU: 3; 
Timestamp: 0:06:11.855817376; Bps: 33416; fps: 25.7; CPU: 0;

NVIDIA Tegra X1

RTSP sink was tested on the NVIDIA Tegra X1 board, with the IMX219 Sony sensor. Using perf you can get CPU load, BPS and FPS, as shown below:

Pipeline

gst-launch-1.0 nvcamerasrc sensor-id=0 ! omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string) \
byte-stream'  ! h264parse  ! 'video/x-h264, mapping=/stream1' ! perf ! rtspsink service=5000

FPS

Timestamp: 3:59:14.941918568; Bps: 28443; fps: 30.3
Timestamp: 3:59:15.943461170; Bps: 32116; fps: 29.97
Timestamp: 3:59:16.949348157; Bps: 27899; fps: 29.85
Timestamp: 3:59:17.953943021; Bps: 29440; fps: 29.88
Timestamp: 3:59:18.959193504; Bps: 28735; fps: 29.85
Timestamp: 3:59:19.960098613; Bps: 25735; fps: 30.0
Timestamp: 3:59:20.965807217; Bps: 27132; fps: 29.85
Timestamp: 3:59:21.967152009; Bps: 27030; fps: 29.97

CPU load

Measured with tegrastats application:

RAM 1016/3994MB (lfb 475x4MB) cpu [40%,12%,14%,25%]@307 GR3D 0%@76 EDP limit 0
RAM 1016/3994MB (lfb 475x4MB) cpu [37%,18%,12%,32%]@307 GR3D 0%@76 EDP limit 0
RAM 1016/3994MB (lfb 475x4MB) cpu [38%,19%,22%,26%]@204 GR3D 0%@76 EDP limit 0
RAM 1016/3994MB (lfb 475x4MB) cpu [34%,35%,18%,10%]@307 GR3D 0%@76 EDP limit 0
RAM 1016/3994MB (lfb 475x4MB) cpu [33%,34%,18%,29%]@204 GR3D 0%@76 EDP limit 0
RAM 1016/3994MB (lfb 475x4MB) cpu [19%,27%,23%,38%]@307 GR3D 0%@76 EDP limit 0
RAM 1016/3994MB (lfb 475x4MB) cpu [32%,30%,13%,32%]@204 GR3D 0%@76 EDP limit 0

Possible client applications

Being, RTSP compliant, gst-rtsp-sink may be received using multiple media clients. The table below exemplifies some of the most popular media client applications and how to receive an RTSP stream produced by the following pipeline:

 gst-launch-1.0 videotestsrc is-live=true ! x264enc ! video/x-h264,mapping=test ! rtspsink service=5000
Client Application Example
VLC vlc rtsp://127.0.0.1:5000/test
MPlayer mplayer rtsp://127.0.0.1:5000/test
Totem totem rtsp://127.0.0.1:5000/test
GstPlay gst-play-1.0 rtsp://127.0.0.1:5000/test
GstLaunch (Playbin) gst-launch-1.0 playbin uri=rtsp://127.0.0.1:5000/test
GstLaunch (RtspSrc) gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:5000/test ! decodebin ! autovideosink

Additional documentation

Access the full plugin documentation by making a gst-inspect of the rtspsink

gst-inspect rtspsink

Legacy RTSP Sink

The old version of RTSP Sink maintains the same idea of integrating an RTSP server into a gstreamer pipeline but its developed to support a single video stream. The main difference is that the service is specified by means of a property rather than on the caps.

On target board

gst-launch videotestsrc ! dmaienc_h264 idrinterval=30 targetbitrate=2500000 single-nalu=false ! rtspsink mapping="/mystream" service=554

On host PC

On the host PC you might need to use root privileges to access the respective sockets.

sudo gst-launch videotestsrc ! x264enc ! rtspsink mapping="/mystream" service=554

Client

As an RTSP client vlc may be used as the following:

vlc rtsp://<target.or.host.pc.ip.address>:554/mystream

See also