Thundercomm TurboX C8550 - GStreamer Pipelines

Thundercomm TurboX C8550
Introduction
SoM Overview Why the TurboX C8550? Carrier Boards Thundercomm TurboX C8550 Developer Kit
Software Development Support
Boot Devices Thundercomm SDK Installing SDK Manager Configuring SDK Manager Downloading Prebuilt Images Building Images from Source Flashing the Images Special / Debug Modes Setting up Board in 9008 Mode Setting up Board in Fastboot Mode
Development in the Board
Getting into the Board Using ADB Using Serial Console Using SSH Device Setup Ethernet Setup WiFi Setup HDMI Setup Camera Setup
Intelligent SDK
QIM SDK TFLite SDK
GStreamer Pipelines
Contact Us

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.

GStreamer is a multimedia framework based in pipelines that link together a variety of media processing systems. It enables quick prototyping of multimedia projects . In this section, we will see some examples of pipelines to capture from the camera on the Thundercomm TurboX C8550 and how to work with it for different applications such as encoding, recording, inference, streaming, among others. This section requires to have installed the QIM SDK, so make sure you have followed the steps for installing it the QIM SDK section.

Note

NOTE: All the pipelines described in this section were tested using GStreamer version 1.20.7.

Performance Measurements

The performance measurements are extracted with the GStreamer pipeline running in the system.

CPU Usage

• For the CPU percentage utilization, we use the top command along with the gst-perf plugin from RidgeRun. By enabling the print-cpu-load property on the perf element it will print the average load percentage on the CPU.

FPS

• For the FPS, we also use the gst-perf plugin from RidgeRun. The plugin computes the FPS mean. The following example shows a pipeline using the perf element:

gst-launch-1.0 videotestsrc ! "video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1" ! perf print-cpu-load=true ! fakesink -v

Glass-to-glass Latency

For this method, we use a stopwatch. The camera from the TurboX C8550 is setup to look at the stopwatch and we measure the time difference from the actual stopwatch time to the time displayed in the monitor.

Pipelines

Note

All the camera pipelines described in this section were tested using the IMX577 camera. The OV9282 tracking camera is currently not supported by the latest LE release.. To learn how to connect the camera please check the Camera Setup section.

Before running the following pipelines make sure you have executed the following commands:

adb root
adb remount
adb shell mount -o remount,rw /
adb shell
echo halBufferMgrMode=0 >> /vendor/etc/camera/camxoverridesettings.txt

Camera to Display

Note

To show the contents of the streams on the display is required to export the XDG_RUNTIME_DIR and WAYLAND_DISPLAY variables.

• 1920x1080@30FPS

export XDG_RUNTIME_DIR=/run/user/root
export WAYLAND_DISPLAY=wayland-1
gst-launch-1.0 qtiqmmfsrc name=qmmf camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! waylandsink

• 1280x720@60FPS

export XDG_RUNTIME_DIR=/run/user/root
export WAYLAND_DISPLAY=wayland-1
gst-launch-1.0 qtiqmmfsrc name=qmmf camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1280,height=720,framerate=60/1 ! queue ! waylandsink

Table 1: Performance of camera IMX577 to display.

Operation Mode	CPU (%)	FPS	latency (ms)
1920x1080@30FPS	14.9	29.996	117
1280x720@60FPS	19.5	59.998	82

Encoding

• 1920x1080@30FPS H264

gst-launch-1.0 qtiqmmfsrc name=camsrc camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! qtic2venc ! h264parse ! mp4mux ! queue ! filesink location="/data/video.mp4" -e

• 1280x720@60FPS H264

gst-launch-1.0 qtiqmmfsrc name=camsrc camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1280,height=720,framerate=60/1 ! queue ! qtic2venc ! h264parse ! mp4mux ! queue ! filesink location="/data/video.mp4" -e

• 1920x1080@30FPS H265

gst-launch-1.0 qtiqmmfsrc name=camsrc camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! qtic2venc ! h265parse ! mp4mux ! queue ! filesink location="/data/video.mp4" -e

• 1280x720@60FPS H265

gst-launch-1.0 qtiqmmfsrc name=camsrc camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1280,height=720,framerate=60/1 ! queue ! qtic2venc ! h265parse ! mp4mux ! queue ! filesink location="/data/video.mp4" -e

Decoding

For these tests the videos are of 1 minute and were recorded using the pipelines from the previous section.

• 1920x1080@30FPS H264

export XDG_RUNTIME_DIR=/run/user/root
export WAYLAND_DISPLAY=wayland-1
FILE=/data/video_h264.mp4
gst-launch-1.0 filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! qtic2vdec ! waylandsink

• 1920x1080@30FPS H265

export XDG_RUNTIME_DIR=/run/user/root
export WAYLAND_DISPLAY=wayland-1
FILE=/data/video_h265.mp4
gst-launch-1.0 filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! qtic2vdec ! waylandsink

Table 2: Performance of file decoding to display.

Operation Mode	CPU (%)	FPS
1920x1080@30FPS H264	3.1	30.017
1280x720@60FPS H264	5.1	59.984
1920x1080@30FPS H265	3.1	29.983
1280x720@60FPS H265	5.2	59.992

Composition

This pipeline composes a stream of the camera (1920x1080@30FPS) with a video recording (1920x1080@30FPS) on the bottom right corner of the final image.

export XDG_RUNTIME_DIR=/run/user/root
export WAYLAND_DISPLAY=wayland-1
FILE=/data/video.mp4
gst-launch-1.0 -e qtivcomposer name=mixer sink_0::position="<0, 0>" sink_0::dimensions="<1920, 1080>" sink_1::position="<1280, 720>" sink_1::dimensions="<640, 360>" ! queue ! waylandsink fullscreen=true \
qtiqmmfsrc name=camsrc camera=2 ! video/x-raw\(memory:GBM\),width=1920,height=1080,framerate=30/1 ! queue ! mixer. \
filesrc location=$FILE ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! qtic2vdec ! queue ! mixer.

Table 3: Performance of camera video composition to display.

CPU (%)	FPS
19.9	30.211

Inference

TFLite + YOLOv5

This pipeline will detect objects and display the bounding boxes using the qtioverlay plugin.

export XDG_RUNTIME_DIR=/run/user/root
export WAYLAND_DISPLAY=wayland-1
setprop persist.overlay.use_c2d_blit 2
gst-launch-1.0 -e qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true qtiqmmfsrc name=qmmf camera=2 ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! metamux. split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=gpu model=/data/yolov5m-320x320-int8.tflite ! queue ! qtimlvdetection threshold=75.0 results=10 module=yolov5 labels=/data/yolov5m.labels constants="YoloV5,q-offsets=<3.0>,q-scales=<0.005047998391091824>;" ! text/x-raw ! queue ! metamux.

Table 4: Performance of camera inference to display.

CPU (%)	FPS
22	18.829