GStreamer Pan Tilt Zoom and Rotate Element - RB5/RB6 Pipelines

GStreamer PTZR
GstPTZR Basics
PTZR Camera Digital PTZR Emulation
User Guide
Pan & Tilt Zoom Rotate Rectangle
Getting Started
Evaluating GstPTZR Getting the code Building GstPTZR
Examples
PC pipelines iMX6 pipelines iMX8 pipelines Tegra pipelines RB5/RB6 pipelines Demo Tool
Contact Us

This page provides example GStreamer pipelines to test GstPTZR features on a Qualcomm RB5/RB6. These pipelines were tested on a Qualcomm RB5 with LU (Linux Ubuntu 20.04).

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.

Note

Before starting, follow the steps for bringing up the display from our Qualcomm RB5 Developer Wiki. Moreover, the compilation must be set to the platform : snapdragon.

Environment

Please, enable the HDMI with the following instructions: Qualcomm_Robotics_RB5/Development_in_the_Board/Device_Setup/HDMI_Setup and set the following environment variables (if you are remotely connected to the RB5):

export WAYLAND_DISPLAY=wayland-0
export XDG_RUNTIME_DIR=/usr/bin/weston_socket

GStreamer pipelines

Same input and output resolution with zoom level 1:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'

gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

You can change the zoom level by changing "zoom-level".

Magnification (zoom level > 1):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=2 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Reduction (zoom level < 1)

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=0 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

You can perform translations relative to the center of the input image by setting the "pan-level" and "tilt-level" properties or perform translations relative to the current position with "pan-delta" and "tilt-delta".

Translate right (-1 < pan level < 0):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr pan-level=-1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Translate left (0 < pan level < 1):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr pan-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Translate down (-1 < tilt level < 0):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr tilt-level=-1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Translate up (0 < tilt level < 1):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr tilt-level=100 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

The property "normalize-translation" (true by default) lets you switch the translation normalization ON and OFF. When translation normalization is true the minimum and maximum translation values are mapped to -1 and 1. On the other hand, when it is set to false, the pan and tilt levels reflect pixels on the input image.

Turn off translation normalization:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr normalize-translation=false ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

The property "output-reference" (true by default) lets you change the coordinate system used as reference for translations. This is specially useful after performing a rotation. If you want to move over the input image axis, it must be changed to false.

Set input coordinate system as reference for translations:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr output-reference=false ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

To rotate the input modify the "rotate-level" property (degrees). You can also perform rotation relative to the current angle with "rotate-delta":

Rotate counter-clockwise (rotate level < 0):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr rotate-level=-100 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Rotate clockwise (rotate level > 0):

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr rotate-level=100 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

The element supports input and output of any resolution and aspect ratio

Smaller input image, different aspect ratio:

IN_CAPS='video/x-raw,format=RGBA,width=1024,height=768,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Bigger input image, different aspect ratio:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1280,height=1024,framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

You can use ranges for input and output caps or leave them unspecified

Resolution range:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=[480,1280],height=[680,1024],framerate=30/1'
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Unspecified caps:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! fpsdisplaysink silent=0 video-sink=fakesink -vvv

Gstd pipelines

Run the element with gstd to change parameters dynamically:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1600,height=900,framerate=30/1'
gstd&
gstd-client pipeline_create p0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr name=ptzr0 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
gstd-client pipeline_play p0

Change propierties:

gstd-client element_set p0 "ptzr0 zoom-level 0.0" 
gstd-client element_set p0 "ptzr0 zoom-level 2.0" 
gstd-client element_set p0 "ptzr0 zoom-level 1.0" 
gstd-client element_set p0 "ptzr0 tilt-level -1.0" 
gstd-client element_set p0 "ptzr0 tilt-level 1.0" 
gstd-client element_set p0 "ptzr0 tilt-level 0.0" 
gstd-client element_set p0 "ptzr0 pan-level -1.0" 
gstd-client element_set p0 "ptzr0 pan-level 1.0" 
gstd-client element_set p0 "ptzr0 pan-level 0.0" 
gstd-client element_set p0 "ptzr0 tilt-delta -1.0" 
gstd-client element_set p0 "ptzr0 tilt-delta 1.0" 
gstd-client element_set p0 "ptzr0 tilt-delta 0.0" 
gstd-client element_set p0 "ptzr0 pan-delta -1.0" 
gstd-client element_set p0 "ptzr0 pan-delta 1.0" 
gstd-client element_set p0 "ptzr0 pan-delta 0.0" 
gstd-client element_set p0 "ptzr0 rotate-level -90.0" 
gstd-client element_set p0 "ptzr0 rotate-level 90.0" 
gstd-client element_set p0 "ptzr0 rotate-level 0.0" 
gstd-client element_set p0 "ptzr0 rotate-delta -90.0" 
gstd-client element_set p0 "ptzr0 rotate-delta 90.0" 
gstd-client element_set p0 "ptzr0 rotate-delta 0.0" 
gstd-client element_set p0 "ptzr0 normalize-translation false" 
gstd-client element_set p0 "ptzr0 normalize-translation true" 
gstd-client element_set p0 "ptzr0 output-reference false" 
gstd-client element_set p0 "ptzr0 output-reference true"

Stop the pipeline:

gstd-client pipeline_stop p0
gstd-client pipeline_delete p0

Performance

Performance measured by using:

# Average Behaviour
gst-launch-1.0 qtiqmmfsrc ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! qtivtransform ! queue ! ptzr name=ptzr0 zoom-level=3 tilt-level=0.5 pan-level=0.5 rotate-level=30 ! perf print-cpu-load=1 ! queue ! fakesink

# Limit Performance
gst-launch-1.0 videotestsrc num-buffers=1 ! imagefreeze ! "video/x-raw,format=RGBA,width=1280,height=720" ! queue ! ptzr name=ptzr0 zoom-level=3 tilt-level=0.5 pan-level=0.5 rotate-level=30 ! perf print-cpu-load=1 ! queue ! fakesink

Resolution	CPU Load (overall)	RAM [MiB]	Framerate
720p	10	52.43	139.8
1080p	12	72.60	102.2
4K	15	159.96	23.5

The CPU consumption might be lower depending on the framerate and camera connection.