Introduction to network streaming using GStreamer
From RidgeRun Developer Connection
THIS PAGE IS UNDER CONSTRUCTION
This document is a basic guide to understand how to use GStreamer for network transmissions using the LeopardBoard DM365.
Basics of GStreamer and network streaming
GStreamer is a framework for multimedia applications that allows to you to create multimedia applications. You can find a detailed information about GStreamer in the GStreamer Application Development Manual. Basically GStreamer works by using the elements concept, that is, it uses a group of plugins each one is designed to do a specific function, for example there are plugins that encode/decode video in a specific format.
Each element has one or more sink paths and one or more source paths, these paths allows to input a media flow into an element and keep the flow to others elements when the data is dropout from the source path. There are some elements that only have source paths (called source elements) , they supply data to the pipeline; also there are sink elements which only have sink paths, they act as sink for the data such as a file or the ALSA sound system.
One or more elements that executes a specific function is called a pipeline. A pipeline could be seem as an element by itself with source and sink paths. In Figure 1 is shown a basic example of a pipeline, in this case all elements are chained in order to act as a OGG files player.
When you work in stream application you will find a pipeline structure likely the shown in the Figure 2.
A basic video streaming example using the LeopardBoard DM365
Making a video streaming from camera sensor at 720P
Network streaming using SDP files
Session Description Protocol(SDP) files are simple text files describing multimedia sessions, in other words, this files advertise the type and characteristics of the session. Since GStreamer can be used for network streaming, programs like VLC can be used to capture this media stream using a SDP file.
Generating a SDP file from a streaming pipeline
Adding the parameter "-v" to a GStreamer pipeline will print the caps negotiated by the sink element. For instance :
-Video caps /GstPipeline:pipeline0/GstUDPSink:udpsink1.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, \ sprop-parameter-sets=(string)\"Z2QAKK2EBUViuKxUdCAqKxXFYqOhAVFYrisVHQgKisVxWKjoQFRWK4rFR0ICorFcVio6ECSFITk8nyfk/k/J8nm5s00IEkKQnJ5Pk/J/J+T5PNzZprQFoeyA\\,aO48sA\\=\\=\", \ payload=(int)96, ssrc=(uint)2997376707, clock-base=(uint)1174645084, seqnum-base=(uint)32878 -Audio caps /GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)MP4A-LATM, \ cpresent=(string)0, config=(string)40002b20, payload=(int)96, ssrc=(uint)2197000970, clock-base=(uint)2213944731, seqnum-base=(uint)24697
Using this caps you can create an SDP file used to capture this media stream using the following mapping
v=<version> o= <owner> IN IP4 <IP4 ADDRESS> c=IN IP4 <IP4 ADDRESS> s=<STREAM "HUMAN" DESCRIPTION> m=<media> <udp port> RTP/AVP <payload> a=rtpmap:<payload> <encoding-name>/<clock-rate>[/<encoding-params>] a=fmtp:<payload> <param>=<value>;...
A full example that includes a H264+AAC encoding and streaming pipeline with its SDP file is shown below (tested using VLC):
HOST_IP=126.96.36.199 AUDIO_PORT=5008 VIDEO_PORT=5004 gst-launch -e alsasrc ! audio/x-raw-int,rate=8000 ! queue ! dmaienc_aac bitrate=56000 ! rtpmp4apay ! udpsink host=$AUDIO_IP port=$HOST_PORT v4l2src always-copy=false input-src=composite ! "video/x-raw-yuv, width=720, height=480, format=(fourcc)NV12, pitch=736" ! dmaiaccel ! dmaienc_h264 ratecontrol=4 encodingpreset=2 ! queue ! rtph264pay ! udpsink host=188.8.131.52 port=$VIDEO_PORT -v
The matching SDP file (once you set the IN IP4 address for your system):
v=0 o=- 1208520720 2590316915 IN IP4 184.108.40.206 c=IN IP4 220.127.116.11 s=ESP H264+AAC STREAM m=video 5004 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 media=video; clock-rate=90000; encoding-name=H264; sprop-parameter-sets=Z2QAKK2EBUViuKxUdCAqKxXFYqOhAVFYrisVHQgKisVxWKjoQFRWK4rFR0ICorFcVio6ECSFITk8nyfk/k/J8nm5s00IEkKQnJ5Pk/J/J+T5PNzZprQFoeyA,aO48sA== a=control:trackID=1 m=audio 5008 RTP/AVP 96 a=rtpmap:96 MP4A-LATM/8000 a=fmtp:96 media=audio; clock-rate=8000; encoding-name=MP4A-LATM; cpresent=0; config=40002b20; payload=96 a=control:trackID=2
Playing the media stream using VLC
Using VLC to capture the network streaming is pretty straightforward:
Other sources of information
Further information about SDP files and GStreamer can be found here :