LeopardBoard GStreamer Pipelines

From RidgeRun Developer Connection

Revision as of 18:25, 1 September 2011 by Pbarrantes (Talk | contribs)
Jump to:navigation, search

The following examples show the usage of GStreamer with the RR DM365 Leopard board SDK 2011Q2. All the commands indicated in this examples are intended to be executed on the target, NOT the host.

Contents

Modifying audio levels

The sound should be heard on the headphones/speakers If the audio volume is too low, edit it by typing:

 / # alsamixer
Then increase the PCM and Line DAC levels.
Also if the sound is noisy activate the AGC option using alsamixer (key M toggles between mute and active).
lqqqqqqqqqqqqqqqqqqqqqqqqqqqq AlsaMixer v1.0.24.2 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Card: DaVinci DM365 EVM                              F1:  Help               x
x Chip:                                                F2:  System information x
x View: F3:[Playback] F4: Capture  F5: All             F6:  Select sound card  x
x Item: PCM [dB gain: -23.50, -23.50]                  Esc: Exit               x
x                                                                              x
x     lqqk     lqqk              lqqk     lqqk     lqqk              lqqk      x
x     x  x     x  x              x ax     x  x     x ax              xa x      x
x     x  x     x  x              x ax     x  x     x ax              xa x      x
x     x  x     x  x              x ax     x  x     x ax              xa x      >
x     x  x     x  x              x ax     x  x     x ax              xa x      >
x     x  x     x  x              x ax     x  x     x ax              xa x      >
x     x  x     x  x              x ax     x  x     x ax              xa x      >
x     x  x     x  x              x ax     x  x     x ax              xa x      >
x     xaax     xaax              xaax     xaax     xaax              xaax      >
x     xaax     xaax              xaax     xaax     xaax              xaax      >
x     xaax     xaax              xaax     xaax     xaax              xaax      x
x     xaax     xaax              xaax     xaax     xaax              xaax      x
x     mqqj     mqqj     lqqk     mqqj     mqqj     mqqj     lqqk     mqqj      x
x                       xOOx                                xOOx               x
x                       mqqj                                mqqj               x
x    35<>35   41<>41            41<>100    41     41<>100          100<>41     x
x  <  PCM   >Line DAC  LineL   LineL DA LineL Le LineL Li  LineR   LineR Li    x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

Testing Audio Input

Audio Test Source

Connect an audio player device to the MIC jack and speakers/headphones to the OUT jack. We did not test with an actual microphone. Play some audio on the device and execute the next pipeline

gst-launch audiotestsrc ! alsasink sync =false

Routing Line In to Line Out

gst-launch alsasrc ! alsasink sync=false

Capturing Audio to AAC File

gst-launch -e alsasrc ! 'audio/x-raw-int,rate=(int)44100,channels=(int)2' ! dmaienc_aac bitrate=128000 ! qtmux ! filesink location=audio_aac.mp4

Capturing Audio to MP3 File

For this pipeline the MP3 codec and plugin need to be installed, further instructions can be found here
https://www.ridgerun.com/developer/wiki/index.php/Add_MP3_support_to_20011q2_SDK

gst-launch -e alsasrc ! 'audio/x-raw-int,rate=(int)44100,channels=(int)2' ! dmaienc_mp3 bitrate=128000 ! filesink location=audio_mp3.mp3

Enabling video window

When you boot, the OSD (on screen display), called frame buffer, has priority for the video output. To enabled the video window to be visible, you need to run:

fbset -disable

Video and audio playback

Get a demo clip

wget -c http://downloads.dvdloc8.com/trailers/divxdigest/simpsons_movie_trailer.zip

Then unzip it

 unzip simpsons_movie_trailer.zip

And play it

 
gst-launch filesrc location= The\ Simpsons\ Movie\ -\ Trailer.mp4 ! qtdemux name=demux ! queue ! \
 dmaidec_h264 numOutputBufs=16 ! priority nice=-10 !  queue ! priority nice=-10 ! dmaiperf ! TIDmaiVideoSink  \
 accelFrameCopy=true videoOutput=component videoStd=720P_60  demux.audio_00 !queue ! priority nice=-5 ! dmaidec_aac ! alsasink

Video playback

The previewer and resizer for video capture must be initialized by the user. You can do that in two ways:

1. Compile and install the ipiped application located in the File System Configuration->Select target's file system software section of the configuration screen (make config). This application allows to get control over some video capturing properties (for more information see Auto exposure and auto white balance library). You can initialized the previewer and resizer by running the following command:

ipipe-client run-config-script dm365_mt9p031_config

or you can run this command instead:

ipipe-client set-previewer-mode cont

NOTE: The initialization must be done before any video capture session.


2. The V4L2 source element (v4l2src) as a build in property called chain-ipipe which automatically initializes the previewer when it is set (by default it is set to true).

NOTE: when using one of the methods described above the other one must not be used since this can create a conflict when the code tries to re-initialize the previewer.

Video test source

 
gst-launch videotestsrc ! TIDmaiVideoSink sync=false enable-last-buffer=false

Output:DVI, mode: 720P_60

Output:composite, mode: NTSC

 
gst-launch -e v4l2src chain-ipipe=true always-copy=false ! dmaiaccel ! video/x-raw-yuv,format=\(fourcc\)NV12, width=640, height=480, framerate=\(fraction\)30/1 !dmaiperf\
 ! TIDmaiVideoSink sync=false accelFrameCopy=false videoOutput=composite videoStd=D1_NTSC sync=false enable-last-buffer=false&

Output:composite, mode: NTSC with Auto exposure white balance demo library

ipipe-client run-config-script dm365_mt9p031_config
ipipe-client init-aew G EC S C 150000 15 50 640 480 50
gst-launch -e v4l2src chain-ipipe=false always-copy=false ! dmaiaccel ! video/x-raw-yuv,format=\(fourcc\)NV12, width=640, height=480, framerate=\(fraction\)30/1 ! dmaiperf\
 ! TIDmaiVideoSink sync=false accelFrameCopy=false videoOutput=composite videoStd=D1_NTSC sync=false enable-last-buffer=false&

Video Recording

Recording h264 NTSC

 
gst-launch -e v4l2src always-copy=true chain-ipipe=true num-buffers=300 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=640,height=480' ! dmaiaccel !\
 dmaienc_h264 encodingpreset=2 ratecontrol=2 intraframeinterval=23 idrinterval=46 targetbitrate=3000000 !dmaiperf ! qtmux ! filesink location=test8003.h264\
 sync=false enable-last-buffer=false&

Recording h264 NTSC with Auto exposure white balance demo library

ipipe-client run-config-script dm365_mt9p031_config
ipipe-client init-aew G EC S C 150000 15 50 640 480 50
gst-launch -e v4l2src always-copy=true chain-ipipe=false num-buffers=300 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=640,height=480' ! dmaiaccel ! dmaienc_h264\
 encodingpreset=2 ratecontrol=2 intraframeinterval=23 idrinterval=46 targetbitrate=3000000 ! dmaiperf ! qtmux ! filesink location=test8003.h264 sync=false\
 enable-last-buffer=false&

Recording h264 1280x720

 
gst-launch -e v4l2src always-copy=FALSE chain-ipipe=true num-buffers=300 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720' ! dmaiaccel !dmaienc_h264\
 encodingpreset=2 ratecontrol=2 intraframeinterval=23 idrinterval=46 targetbitrate=3000000 ! dmaiperf ! qtmux ! filesink location=test.h264 sync=false enable-last-buffer=false&

Recording h264 1280x720 with Auto exposure white balance demo library

ipipe-client run-config-script dm365_mt9p031_config
ipipe-client init-aew G EC S C 150000 15 50 1280 720 50
gst-launch -e v4l2src always-copy=FALSE chain-ipipe=false num-buffers=300 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=1280,height=720' ! dmaiaccel !dmaienc_h264 encodingpreset=2 ratecontrol=2 intraframeinterval=23 idrinterval=46 targetbitrate=3000000 ! dmaiperf\
 ! qtmux ! filesink location=test.h264 sync=false enable-last-buffer=false&

Video recording and decoding (NOT TESTED)

This section captures video from an NTSC source and an audio source on the line-in connector. For every encoding pipeline it's complementary decoding pipeline is shown below

H264+AAC

gst-launch -e v4l2src always-copy=FALSE input-src=composite ! "video/x-raw-yuv,format=(fourcc)NV12,width=720,height=480,pitch=736" ! \
dmaiaccel ! dmaienc_h264 encodingpreset=2 targetbitrate=3600000 outputBufferSize=2000000 ! dmaiperf print-arm-load=true ! queue ! \
qtmux name=m alsasrc buffer-time=800000 latency-time=30000 ! 'audio/x-raw-int,rate=22050' ! dmaienc_aac outputBufferSize=131072 bitrate=128000 ! \
m. m. ! filesink location= av_h264.mp4
gst-launch filesrc location= av_h264.mp4 ! qtdemux name=demux ! queue ! dmaidec_h264 numOutputBufs=18 ! queue ! dmaiperf ! \
TIDmaiVideoSink accelFrameCopy=true videoOutput=composite videoStd=D1_NTSC  demux.audio_00 ! queue ! dmaidec_aac ! alsasink

MPEG2+MP3

For this pipeline the MP3 codec and plugin need to be installed, further instructions can be found here
https://www.ridgerun.com/developer/wiki/index.php/Add_MP3_support_to_20011q2_SDK

gst-launch -e alsasrc latency-time=30000 buffer-time=800000 ! "audio/x-raw-int, rate=(int)22050" ! dmaienc_mp3 bitrate=128000 outputBufferSize=131072 ! \
queue ! mpegpsmux name=m v4l2src always-copy=false input-src=composite ! "video/x-raw-yuv,format=(fourcc)NV12,width=720,height=480,pitch=736" ! \
dmaiaccel ! dmaienc_mpeg2 encodingpreset=2 targetbitrate=3600000 ! dmaiperf print-arm-load=true ! queue ! m. m. ! \
filesink location= av_mpeg2.mpg
gst-launch filesrc location= av_mpeg2.mpg ! mpegdemux name=demux .audio_00 ! queue ! mad ! alsasink demux.video_00 ! queue ! dmaidec_mpeg2 ! \
TIDmaiVideoSink accelFrameCopy=true videoOutput=composite videoStd=D1_NTSC

MJPEG+AAC

gst-launch -e alsasrc provide-clock=false latency-time=30000 buffer-time=800000 ! 'audio/x-raw-int,rate=(int)16000,channels=(int)2' ! \
dmaienc_aac bitrate=128000 ! queue ! qtmux name=m v4l2src always-copy=false input-src=composite ! \
'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=480,pitch=736' ! dmaiaccel ! dmaienc_mjpeg qValue=30 outputBufferSize=2500000 ! \
dmaiperf print-arm-load=true !  queue ! m. m. ! filesink location=test_av_mjpeg.mp4
gst-launch filesrc location = /mnt/MSD_mmcblk0p1/test_av_mjpeg.mp4 ! qtdemux name=demux .video_00 ! queue ! dmaidec_mjpeg ! \
'video/x-raw-yuv, format=(fourcc)NV12' ! TIDmaiVideoSink videoOutput=composite videoStd=D1_NTSC demux.audio_00 ! queue ! \
dmaidec_aac ! alsasink

Camera streaming over the network

These instructions show how to do video streaming over the network, a video will be played on the board and viewed on the host.

H264 video streaming pipelines LeopardBoard DM365

HOST_ADDR=<Client's IP address>
PORT=3000

gst-launch v4l2src always-copy=FALSE chain-ipipe=true ! video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! queue ! dmaiaccel ! dmaienc_h264 encodingpreset=2 ratecontrol=2 intraframeinterval=23 idrinterval=46 targetbitrate=3000000\
 ! rtph264pay !udpsink port=$PORT host=$HOST_ADDR sync=false enable-last-buffer=false&
PORT=3000

gst-launch udpsrc port=$PORT ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, profile-level-id=(string)640028, sprop-parameter-sets=(string)"J2QAKK2IDkOYIOEMKQpEByHMEHCGFIUiA5DmCDhDCkKQwEIYwhxmMhCGAhDGEOMxkIQwEIYwhxmMhCICEZjOI8KfEfiP4j8R8R4ziMREQoEIjEcR4j5PxH8n5PiPEcRkiLQCgC3I\,KO48sA\=\=", payload=(int)96, ssrc=(guint)1646914791, clock-base=(guint)4182448954, seqnum-base=(guint)19314' ! rtph264depay ! queue ! ffdec_h264 ! xvimagesink sync=false

Camera streaming over the network with Auto exposure white balance demo library

ipipe-client run-config-script dm365_mt9p031_config
ipipe-client init-aew G EC S C 150000 15 50 1280 720 50
HOST_ADDR=<Client's IP address>
PORT=3000
gst-launch v4l2src always-copy=FALSE chain-ipipe=false ! video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! queue\
 ! dmaienc_h264  dmaienc_h264 encodingpreset=2 ratecontrol=2 intraframeinterval=23 idrinterval=46 targetbitrate=3000000 ! rtph264pay ! udpsink port=$PORT host=$HOST_ADDR sync=false enable-last-buffer=false&
PORT=3000

gst-launch udpsrc port=$PORT ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, profile-level-id=(string)640028, sprop-parameter-sets=(string)"J2QAKK2IDkOYIOEMKQpEByHMEHCGFIUiA5DmCDhDCkKQwEIYwhxmMhCGAhDGEOMxkIQwEIYwhxmMhCICEZjOI8KfEfiP4j8R8R4ziMREQoEIjEcR4j5PxH8n5PiPEcRkiLQCgC3I\,KO48sA\=\=", payload=(int)96, ssrc=(guint)1646914791, clock-base=(guint)4182448954, seqnum-base=(guint)19314' ! rtph264depay ! queue ! ffdec_h264 ! xvimagesink sync=false

Image capture

You also can capture JPEG images from the camera module. The following pipelines capture a JPEG image for VGA and 720P resolutions using the MT9P031 sensor.

FILE_NAME=VGA_jpeg_file.jpg

gst-launch -e v4l2src always-copy=false num-buffers=1 chain-ipipe=true ! video/x-raw-yuv,format=\(fourcc\)UYVY, width=640, height=480 ! dmaienc_jpeg ! filesink location=$FILE_NAME


FILE_NAME=720P_jpeg_file.jpg

gst-launch -e v4l2src always-copy=false num-buffers=1 chain-ipipe=true ! video/x-raw-yuv,format=\(fourcc\)UYVY, width=1280, height=720 ! dmaienc_jpeg ! filesink location=$FILE_NAME

Additional Examples

Capturing video from external modules

Please see Hardware setup - LeopardBoard DM365 in order to get see more examples of capturing video from external modules.

Navigation
Toolbox