Qualcomm Robotics RB5/RB6 - Decoding GStreamer Pipelines

Qualcomm Robotics RB5/RB6
Introduction
SoM Overview Why the RB5/RB6? Carrier Boards Qualcomm Robotics RB5 developer kit Qualcomm Robotics RB6 platform
Software Development Support
Boot Devices Thundercomm SDK Downloading SDK Installing SDK Generating Image and flashing Linaro Getting Started Flashing Bootloader Flashing Boot Image and Rootfs Special / Debug Modes Setting up board in EDL Mode Setting up board in Fastboot Mode
Development in the Board
Getting into the Board Using ADB Using Serial Console Using SSH Device Setup WiFi setup HDMI setup Update Ubuntu Packages
Build from source
Base Image Customize Image
Capture Subsystem
Hardware Capture Components Software Capture Components Building your own camera sensor driver
Image Processing and Acceleration Software
GPU Profiling Fast CV Installing Hexagon SDK Installing Fast CV Building your own Fast CV application
GStreamer Pipelines
Capture and Display Video Encoding Video Decoding
AI Acceleration
Neural Processing SDK Downloading Requirements Setup SDK Environment Install qtimlesnpe Running Inception v3 Running MobileNetV2 + SSD TensorFlow Example pipeline TensorFlow to DLC Conversion
Sensors
IMU
Demos
Drone Demo Smart Camera
Contact Us

In this section, we will present some GStreamer pipelines to decode video in the Qualcomm Robotics RB5 development kit. We will cover decoding of H264 and H265 and display the video in our monitor. The decoding will be done both by software and hardware, the last one using plugins from both the OpenMAX and QTI library. We also measure key performance indicator for each pipeline. Check our GStreamer Pipelines section to find more information about how we extracted the performance metrics presented in this section. For the H264 pipelines we use a video file of 30 MB of size and for the H265 we use a video of 156 KB.

Hardware Acclerated

The Qualcomm Robotics RB5/RB6 uses OpenMAX to perform hardware accelerated decoding. The board comes with two sets of plugins that will help perform the decoding. One of them is from the QTI library, it is the qtivdec element. This element is a V4L2 based video decoder that uses QTI's video hardware cores for decoding video in both H264 and H265. The other plugins we are going to use are from OpenMAX, they are the omxh264dec and omxh265dec elements.

QTI

Here we are using the qtivdec element to perform hardware accelerated decoding.

H264

The following pipeline decodes an mp4 video coded in H264 and displays it in the monitor. The video has a resolution of 1920x1080 with a framerate of 30fps. Table 1 shows the performance metrics for this pipeline. We use a FILE variable to specify the video file we are decoding.

FILE=/data/HW_H264_camera.mp4 
gst-launch-1.0 -e -v filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue max-size-buffers=1 ! h264parse ! qtivdec ! waylandsink

Table 1: Performance of pipeline for H264 decoding with qtivdec.

Operation Mode	CPU (%)	FPS
Max performance	2.9	30.022

H265

The following pipeline decodes an mp4 video coded in H265 and displays it in the monitor. The video has a resolution of 1920x1080 with a framerate of 30fps. Table 2 shows the performance metrics for this pipeline. We use a FILE variable to specify the video file we are decoding.

FILE=/data/HW_H265_camera.mp4 
gst-launch-1.0 -e -v filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue max-size-buffers=1 ! h265parse ! qtivdec ! waylandsink

Table 2: Performance of pipeline for H265 decoding with qtivdec.

Operation Mode	CPU (%)	FPS
Max performance	3.1	30.315

Decoding using OpenMAX plugins

In here we are decoding video using the OpenMAX plugins that will allow to use its platform for hardware accelerated decoding.

H265

The following pipeline decodes an mp4 video coded in H265 and displays it in the monitor. We now use the OMX plugin for H265 decoding. The video has a resolution of 1920x1080 with a framerate of 30fps. Table 4 shows the performance metrics for this pipeline. We use a FILE variable to specify the video file we are decoding.

FILE=/data/HW_H265_camera.mp4 
gst-launch-1.0 filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! omxh265dec ! waylandsink -e -v

Table 4: Performance of pipeline for H265 decoding with omxh265dec.

Operation Mode	CPU (%)	FPS
Max performance	2.2	25.105

No Hardware accelerated

Now we are going to show some example pipelines that decode video in the H264 and H265 formats without using specific hardware for better performance. For this, we are using GStreamer basic decoding elements.

H264

The following pipeline decodes an mp4 video coded in H264 and displays it in the monitor using the avdec_h264 element. The video has a resolution of 1920x1080 with a framerate of 30fps. Table 5 shows the performance metrics for this pipeline. We use a FILE variable to specify the video file we are decoding.

FILE=/data/HW_H264_camera.mp4
gst-launch-1.0 filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! avdec_h264 ! waylandsink -e -v

Table 5: Performance of pipeline for H264 decoding with avdec_h264 with RTSP source.

Operation Mode	CPU (%)	FPS
Max performance	19.9	30.132

We can also decode video from an RTSP stream. In this pipeline, we use the rtspsrc element to get an RTSP stream. We define the IP address from the stream, the port and mapping. We later convert the RTSP stream into H264 and finally, we decode it and display it on our monitor. Table 6 shows the performance metrics for this pipeline. The following pipeline shows how:

IP_ADDRESS=192.168.1.8
PORT=5000
MAPPING=training
gst-launch-1.0 -v rtspsrc location=rtsp://${IP_ADDRESS}:${PORT}/${MAPPING} ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! "video/x-raw,width=480,height=360,framerate=30/1" ! waylandsink

Table 6: Performance of pipeline for H264 decoding with avdec_h264 with RTSP source.

Operation Mode	CPU (%)	FPS
Max performance	10	27.863

H265

The following pipeline decodes an mp4 video coded in H265 and displays it in the monitor using the avdec_h265 element. The video has a resolution of 1920x1080 with a framerate of 30fps. Table 7 shows the performance metrics for this pipeline. We use a FILE variable to specify the video file we are decoding.

FILE=/data/HW_H265_camera.mp4
gst-launch-1.0 filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! avdec_h265 ! waylandsink -e -v

Table 7: Performance of pipeline for H265 decoding with avdec_h265.

Operation Mode	CPU (%)	FPS
Max performance	18.3	1.262