GStreamer Pan Tilt Zoom and Rotate Element - RB5/RB6 Pipelines
| GStreamer PTZR |
|---|
 |
| GstPTZR Basics |
| User Guide |
| Getting Started |
| Examples |
| Contact Us |
This page provides example GStreamer pipelines to test GstPTZR features on a Qualcomm RB5/RB6. These pipelines were tested on a Qualcomm RB5 with LU (Linux Ubuntu 20.04).
|
 | Note: before starting, follow the steps for bringing up the display from our Qualcomm RB5 Developer Wiki. Moreover, the compilation must be set to the platform snapdragon. |
Environment
Please, enable the HDMI with the following instructions: Qualcomm_Robotics_RB5/Development_in_the_Board/Device_Setup/HDMI_Setup and set the following environment variables (if you are remotely connected to the RB5):
export WAYLAND_DISPLAY=wayland-0 export XDG_RUNTIME_DIR=/usr/bin/weston_socket
GStreamer pipelines
Same input and output resolution with zoom level 1:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
You can change the zoom level by changing "zoom-level".
Magnification (zoom level > 1):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=2 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Reduction (zoom level < 1)
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=0 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
You can perform translations relative to the center of the input image by setting the "pan-level" and "tilt-level" properties or perform translations relative to the current position with "pan-delta" and "tilt-delta".
Translate right (-1 < pan level < 0):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr pan-level=-1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Translate left (0 < pan level < 1):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr pan-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Translate down (-1 < tilt level < 0):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr tilt-level=-1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Translate up (0 < tilt level < 1):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr tilt-level=100 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
The property "normalize-translation" (true by default) lets you switch the translation normalization ON and OFF. When translation normalization is true the minimum and maximum translation values are mapped to -1 and 1. On the other hand, when it is set to false, the pan and tilt levels reflect pixels on the input image.
Turn off translation normalization:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr normalize-translation=false ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
The property "output-reference" (true by default) lets you change the coordinate system used as reference for translations. This is specially useful after performing a rotation. If you want to move over the input image axis, it must be changed to false.
Set input coordinate system as reference for translations:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr output-reference=false ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
To rotate the input modify the "rotate-level" property (degrees). You can also perform rotation relative to the current angle with "rotate-delta":
Rotate counter-clockwise (rotate level < 0):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr rotate-level=-100 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Rotate clockwise (rotate level > 0):
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr rotate-level=100 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
The element supports input and output of any resolution and aspect ratio
Smaller input image, different aspect ratio:
IN_CAPS='video/x-raw,format=RGBA,width=1024,height=768,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Bigger input image, different aspect ratio:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1280,height=1024,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
You can use ranges for input and output caps or leave them unspecified
Resolution range:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=[480,1280],height=[680,1024],framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Unspecified caps:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' gst-launch-1.0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! fpsdisplaysink silent=0 video-sink=fakesink -vvv
Gstd pipelines
Run the element with gstd to change parameters dynamically:
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' OUT_CAPS='video/x-raw,format=RGBA,width=1600,height=900,framerate=30/1' gstd& gstd-client pipeline_create p0 videotestsrc pattern=circular is-live=true num-buffers=-1 ! capsfilter caps="$IN_CAPS" ! ptzr name=ptzr0 ! capsfilter caps="$OUT_CAPS" ! fpsdisplaysink silent=0 video-sink=fakesink -vvv gstd-client pipeline_play p0
Change propierties:
gstd-client element_set p0 "ptzr0 zoom-level 0.0" gstd-client element_set p0 "ptzr0 zoom-level 2.0" gstd-client element_set p0 "ptzr0 zoom-level 1.0" gstd-client element_set p0 "ptzr0 tilt-level -1.0" gstd-client element_set p0 "ptzr0 tilt-level 1.0" gstd-client element_set p0 "ptzr0 tilt-level 0.0" gstd-client element_set p0 "ptzr0 pan-level -1.0" gstd-client element_set p0 "ptzr0 pan-level 1.0" gstd-client element_set p0 "ptzr0 pan-level 0.0" gstd-client element_set p0 "ptzr0 tilt-delta -1.0" gstd-client element_set p0 "ptzr0 tilt-delta 1.0" gstd-client element_set p0 "ptzr0 tilt-delta 0.0" gstd-client element_set p0 "ptzr0 pan-delta -1.0" gstd-client element_set p0 "ptzr0 pan-delta 1.0" gstd-client element_set p0 "ptzr0 pan-delta 0.0" gstd-client element_set p0 "ptzr0 rotate-level -90.0" gstd-client element_set p0 "ptzr0 rotate-level 90.0" gstd-client element_set p0 "ptzr0 rotate-level 0.0" gstd-client element_set p0 "ptzr0 rotate-delta -90.0" gstd-client element_set p0 "ptzr0 rotate-delta 90.0" gstd-client element_set p0 "ptzr0 rotate-delta 0.0" gstd-client element_set p0 "ptzr0 normalize-translation false" gstd-client element_set p0 "ptzr0 normalize-translation true" gstd-client element_set p0 "ptzr0 output-reference false" gstd-client element_set p0 "ptzr0 output-reference true"
Stop the pipeline:
gstd-client pipeline_stop p0 gstd-client pipeline_delete p0
Performance
Performance measured by using:
# Average Behaviour gst-launch-1.0 qtiqmmfsrc ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! qtivtransform ! queue ! ptzr name=ptzr0 zoom-level=3 tilt-level=0.5 pan-level=0.5 rotate-level=30 ! perf print-cpu-load=1 ! queue ! fakesink # Limit Performance gst-launch-1.0 videotestsrc num-buffers=1 ! imagefreeze ! "video/x-raw,format=RGBA,width=1280,height=720" ! queue ! ptzr name=ptzr0 zoom-level=3 tilt-level=0.5 pan-level=0.5 rotate-level=30 ! perf print-cpu-load=1 ! queue ! fakesink
| Resolution | CPU Load (overall) | RAM [MiB] | Framerate |
|---|---|---|---|
| 720p | 10 | 52.43 | 139.8 |
| 1080p | 12 | 72.60 | 102.2 |
| 4K | 15 | 159.96 | 23.5 |
The CPU consumption might be lower depending on the framerate and camera connection.