THIS PAGE IS UNDER CONSTRUCTION

Introduction

This document is a basic guide to understand how to use GStreamer for network transmissions using the LeopardBoard DM365.

Basics of GStreamer and network streaming

GStreamer is a framework for multimedia applications that allows to you to create multimedia applications. You can find a detailed information about GStreamer in the GStreamer Application Development Manual. Basically GStreamer works by using the elements concept, that is, it uses a group of plugins each one is designed to do a specific function, for example there are plugins that encode/decode video in a specific format.

Each element has one or more sink paths and one or more source paths, these paths allows to input a media flow into an element and keep the flow to others elements when the data is dropout from the source path. There are some elements that only have source paths (called source elements) , they supply data to the pipeline; also there are sink elements which only have sink paths, they act as sink for the data such as a file or the ALSA sound system.

One or more elements that executes a specific function is called a pipeline. A pipeline could be seem as an element by itself with source and sink paths. In Figure 1 is shown a basic example of a pipeline, in this case all elements are chained in order to act as a OGG files player.

Figure 1. Basic example of GStreamer pipeline: a OGG player. [1]

When you work in stream application you will find a pipeline structure likely the shown in the Figure 2.

A basic video streaming example using the LeopardBoard DM365

Making a video streaming from camera sensor at 720P

Network streaming using SDP files

Session Description Protocol(SDP) files are simple text files describing multimedia sessions, in other words, this files advertise the type and characteristics of the session. Since GStreamer can be used for network streaming, programs like VLC can be used to capture this media stream using a SDP file.

Generating a SDP file from a streaming pipeline

Adding the parameter "-v" to a GStreamer pipeline will print the caps negotiated by the sink element. For instance :

-Video caps
/GstPipeline:pipeline0/GstUDPSink:udpsink1.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, \
sprop-parameter-sets=(string)\"Z2QAKK2EBUViuKxUdCAqKxXFYqOhAVFYrisVHQgKisVxWKjoQFRWK4rFR0ICorFcVio6ECSFITk8nyfk/k/J8nm5s00IEkKQnJ5Pk/J/J+T5PNzZprQFoeyA\\,aO48sA\\=\\=\", \
payload=(int)96, ssrc=(uint)2997376707, clock-base=(uint)1174645084, seqnum-base=(uint)32878

-Audio caps
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)MP4A-LATM, \
cpresent=(string)0, config=(string)40002b20, payload=(int)96, ssrc=(uint)2197000970, clock-base=(uint)2213944731, seqnum-base=(uint)24697

Using this caps you can create an SDP file used to capture this media stream using the following mapping

v=<version>
o= <owner> IN IP4 <IP4 ADDRESS>
c=IN IP4 <IP4 ADDRESS>
s=<STREAM "HUMAN" DESCRIPTION>
m=<media> <udp port> RTP/AVP <payload>
a=rtpmap:<payload> <encoding-name>/<clock-rate>[/<encoding-params>]
a=fmtp:<payload> <param>=<value>;...

A full example that includes a H264+AAC encoding and streaming pipeline with its SDP file is shown below (tested using VLC):

HOST_IP=68.195.194.250
AUDIO_PORT=5008
VIDEO_PORT=5004

gst-launch -e alsasrc ! audio/x-raw-int,rate=8000 ! queue ! dmaienc_aac bitrate=56000 ! rtpmp4apay ! udpsink host=$AUDIO_IP port=$HOST_PORT v4l2src always-copy=false input-src=composite ! "video/x-raw-yuv, width=720, height=480, format=(fourcc)NV12, pitch=736" ! dmaiaccel ! dmaienc_h264 ratecontrol=4 encodingpreset=2 ! queue ! rtph264pay ! udpsink host=68.195.194.250 port=$VIDEO_PORT -v

The matching SDP file (once you set the IN IP4 address for your system):

v=0
o=- 1208520720 2590316915 IN IP4 68.195.194.250
c=IN IP4 68.195.194.250
s=ESP H264+AAC STREAM
m=video 5004 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 media=video; clock-rate=90000; encoding-name=H264; sprop-parameter-sets=Z2QAKK2EBUViuKxUdCAqKxXFYqOhAVFYrisVHQgKisVxWKjoQFRWK4rFR0ICorFcVio6ECSFITk8nyfk/k/J8nm5s00IEkKQnJ5Pk/J/J+T5PNzZprQFoeyA,aO48sA==
a=control:trackID=1
m=audio 5008 RTP/AVP 96
a=rtpmap:96 MP4A-LATM/8000
a=fmtp:96 media=audio; clock-rate=8000; encoding-name=MP4A-LATM; cpresent=0; config=40002b20; payload=96
a=control:trackID=2

Playing the media stream using VLC

Using VLC to capture the network streaming is pretty straightforward:

vlc <FILE_PATH>

Other sources of information

Further information about SDP files and GStreamer can be found here :

-SDP

-Gstreamer