GStreamer Pan Tilt Zoom and Rotate Element - NVIDIA Jetson Pipelines

Follow us on:

Share This Page

Facebook Twitter LinkedIn WhatsApp Email Copy Link

Close

Send Email Using

Gmail (Web) Thunderbird (default mail app)

Close

GStreamer PTZR
GstPTZR Basics
PTZR Camera Digital PTZR Emulation
User Guide
Pan & Tilt Zoom Rotate Rectangle
Getting Started
Evaluating GstPTZR Getting the code Building GstPTZR
Examples
PC pipelines iMX6 pipelines iMX8 pipelines NVIDIA Jetson pipelines RB5/RB6 pipelines Dragonwing 9075 pipelines Demo Tool PTZR with an Xbox controller
Contact Us

The following pipelines were tested on the Orin series with JP 6 and 7, as well as on Thor.

Videotestsrc to display

Same input and output resolution with zoom level 1:

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr ! nvvidconv ! xvimagesink

You can change the zoom level by changing "zoom-level".

Magnification (zoom level > 1):

 gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr zoom-level=2 ! nvvidconv ! xvimagesink

Reduction (zoom level < 1)

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr zoom-level=-1 ! nvvidconv ! xvimagesink

You can perform translations relative to the center of the input image by setting the "pan-level" and "tilt-level" properties or perform translations relative to the current position with "pan-delta" and "tilt-delta".

Translate right (-1 < pan level < 0):

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr pan-level=-1 ! nvvidconv ! xvimagesink

Translate left (0 < pan level < 1):

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr pan-level=1 ! nvvidconv ! xvimagesink

Translate down (-1 < tilt level < 0):

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr tilt-level=-1 ! nvvidconv ! xvimagesink

Translate up (0 < tilt level < 1):

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr tilt-level=1 ! nvvidconv ! xvimagesink

The property "normalize-translation" (true by default) lets you switch the translation normalization ON and OFF. When translation normalization is true, the minimum and maximum translation values are mapped to -1 and 1. On the other hand, when it is set to false, the pan and tilt levels reflect pixels on the input image.

Turn off translation normalization:

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr normalize-translation=false ! nvvidconv ! xvimagesink

The property "output-reference" (true by default) lets you change the coordinate system used as reference for translations. This is especially useful after performing a rotation. If you want to move over the input image axis, it must be changed to false.

Set input coordinate system as reference for translations:

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr output-reference=false ! nvvidconv ! xvimagesink

To rotate the input modify the "rotate-level" property (degrees). You can also perform rotation relative to the current angle with "rotate-delta":

Rotate counter-clockwise (rotate level < 0):

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr rotate-level=-100 ! nvvidconv ! xvimagesink

Rotate clockwise (rotate level > 0):

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr rotate-level=100 ! nvvidconv ! xvimagesink

The element supports input and output of any resolution and aspect ratio

Smaller input image, different aspect ratio:

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=1280,height=1024,framerate=30/1' ! nvvidconv ! xvimagesink

Bigger input image, different aspect ratio:

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=3840,height=2160,framerate=30/1' ! nvvidconv ! xvimagesink

You can use ranges for input and output caps or leave them unspecified

Resolution range:

gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=[480,1280],height=[680,1024],framerate=30/1' ! nvvidconv ! xvimagesink

Filesrc to display

Same input and output resolution with zoom level 1:

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr ! nvvidconv ! nv3dsink

You can change the zoom level by changing "zoom-level".

Magnification (zoom level > 1):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr zoom-level=2 ! nvvidconv ! nv3dsink

Reduction (zoom level < 1)

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr zoom-level=-1 ! nvvidconv ! nv3dsink

You can perform translations relative to the center of the input image by setting the "pan-level" and "tilt-level" properties or perform translations relative to the current position with "pan-delta" and "tilt-delta".

Translate right (-1 < pan level < 0):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr pan-level=-1 ! nvvidconv ! nv3dsink

Translate left (0 < pan level < 1):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr pan-level=1 ! nvvidconv ! nv3dsink

Translate down (-1 < tilt level < 0):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr tilt-level=-1 ! nvvidconv ! nv3dsink

Translate up (0 < tilt level < 1):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr tilt-level=1 ! nvvidconv ! nv3dsink

The property "normalize-translation" (true by default) lets you switch the translation normalization ON and OFF. When translation normalization is true, the minimum and maximum translation values are mapped to -1 and 1. On the other hand, when it is set to false, the pan and tilt levels reflect pixels on the input image.

Turn off translation normalization:

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr normalize-translation=false ! nvvidconv ! nv3dsink

The property "output-reference" (true by default) lets you change the coordinate system used as reference for translations. This is especially useful after performing a rotation. If you want to move over the input image axis, it must be changed to false.

Set input coordinate system as reference for translations:

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr output-reference=false ! nvvidconv ! nv3dsink

To rotate the input modify the "rotate-level" property (degrees). You can also perform rotation relative to the current angle with "rotate-delta":

Rotate counter-clockwise (rotate level < 0):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr rotate-level=-100 ! nvvidconv ! nv3dsink

Rotate clockwise (rotate level > 0):

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr rotate-level=100 ! nvvidconv ! nv3dsink

The element supports input and output of any resolution and aspect ratio

Smaller input image, different aspect ratio:

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=1280,height=1024,framerate=30/1' ! nvvidconv ! nv3dsink

Bigger input image, different aspect ratio:

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=3840,height=2160,framerate=30/1' ! nvvidconv ! nv3dsink

You can use ranges for input and output caps or leave them unspecified

Resolution range:

gst-launch-1.0 filesrc location=$INPUT_FILE ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=[480,1280],height=[680,1024],framerate=30/1' ! nvvidconv ! nv3dsink

Camera to file

Same input and output resolution with zoom level 1:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

You can change the zoom level by changing "zoom-level".

Magnification (zoom level > 1):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr zoom-level=2 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Reduction (zoom level < 1)

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr zoom-level=-1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

You can perform translations relative to the center of the input image by setting the "pan-level" and "tilt-level" properties or perform translations relative to the current position with "pan-delta" and "tilt-delta".

Translate right (-1 < pan level < 0):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr pan-level=-1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Translate left (0 < pan level < 1):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr pan-level=1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Translate down (-1 < tilt level < 0):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr tilt-level=-1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Translate up (0 < tilt level < 1):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr tilt-level=1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

The property "normalize-translation" (true by default) lets you switch the translation normalization ON and OFF. When translation normalization is true, the minimum and maximum translation values are mapped to -1 and 1. On the other hand, when it is set to false, the pan and tilt levels reflect pixels on the input image.

Turn off translation normalization:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr normalize-translation=false ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

The property "output-reference" (true by default) lets you change the coordinate system used as reference for translations. This is especially useful after performing a rotation. If you want to move over the input image axis, it must be changed to false.

Set input coordinate system as reference for translations:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr output-reference=false ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

To rotate the input modify the "rotate-level" property (degrees). You can also perform rotation relative to the current angle with "rotate-delta":

Rotate counter-clockwise (rotate level < 0):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr rotate-level=-100 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Rotate clockwise (rotate level > 0):

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr rotate-level=100 ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

The element supports input and output of any resolution and aspect ratio

Smaller input image, different aspect ratio:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !   ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=1280,height=1024,framerate=30/1' ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Bigger input image, different aspect ratio:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=3840,height=2160,framerate=30/1' ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

You can use ranges for input and output caps or leave them unspecified

Resolution range:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=RGBA, framerate=30/1' !  ptzr zoom-level=1 ! 'video/x-raw,format=RGBA,width=[480,1280],height=[680,1024],framerate=30/1' ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=ptzr.mp4 -e

Gstd pipelines

Run the element with gstd to change parameters dynamically:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1600,height=900,framerate=30/1'
gstd&
gstd-client pipeline_create p0 nvcamerasrc fpsRange="30.0 30.0" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr name=ptzr0 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink
gstd-client pipeline_play p0

Change propierties:

gstd-client element_set p0 "ptzr0 zoom-level 0.0" 
gstd-client element_set p0 "ptzr0 zoom-level 2.0" 
gstd-client element_set p0 "ptzr0 zoom-level 1.0" 
gstd-client element_set p0 "ptzr0 tilt-level -1.0" 
gstd-client element_set p0 "ptzr0 tilt-level 1.0" 
gstd-client element_set p0 "ptzr0 tilt-level 0.0" 
gstd-client element_set p0 "ptzr0 pan-level -1.0" 
gstd-client element_set p0 "ptzr0 pan-level 1.0" 
gstd-client element_set p0 "ptzr0 pan-level 0.0" 
gstd-client element_set p0 "ptzr0 tilt-delta -1.0" 
gstd-client element_set p0 "ptzr0 tilt-delta 1.0" 
gstd-client element_set p0 "ptzr0 tilt-delta 0.0" 
gstd-client element_set p0 "ptzr0 pan-delta -1.0" 
gstd-client element_set p0 "ptzr0 pan-delta 1.0" 
gstd-client element_set p0 "ptzr0 pan-delta 0.0" 
gstd-client element_set p0 "ptzr0 rotate-level -90.0" 
gstd-client element_set p0 "ptzr0 rotate-level 90.0" 
gstd-client element_set p0 "ptzr0 rotate-level 0.0" 
gstd-client element_set p0 "ptzr0 rotate-delta -90.0" 
gstd-client element_set p0 "ptzr0 rotate-delta 90.0" 
gstd-client element_set p0 "ptzr0 rotate-delta 0.0" 
gstd-client element_set p0 "ptzr0 normalize-translation false" 
gstd-client element_set p0 "ptzr0 normalize-translation true" 
gstd-client element_set p0 "ptzr0 output-reference false" 
gstd-client element_set p0 "ptzr0 output-reference true"

Stop the pipeline:

gstd-client pipeline_stop p0
gstd-client pipeline_delete p0

Pipelines performance on Orin Nano

This section presents some performance and latency results obtained with the Orin Nano running JetPack 7.2. The following pipelines were utilized.

The following tools were used:

• gst-perf
• gst-shark

Pipelines

Videotestsrc HD

GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA, framerate=30/1' !  ptzr ! perf ! fakesink sync=true

Videotestsrc 4K

GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 videotestsrc is-live=true ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=RGBA, framerate=30/1' !  ptzr ! perf ! fakesink sync=true

Filesrc HD

GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 filesrc location=/home/nvidia/carguedas/perf_mtb_video.mp4 ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=RGBA, framerate=30/1' !  ptzr ! perf ! fakesink sync=true

Filesrc 4K

GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 filesrc location=/home/nvidia/carguedas/stabilized_motocross_video.mp4 ! qtdemux ! h264parse ! nvv4l2decoder  ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=RGBA, framerate=30/1' !  ptzr ! perf ! fakesink sync=true

Nvarguscamerasrc

GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)RGBA, framerate=(fraction)30/1' !  ptzr ! perf ! fakesink sync=true

Results

Pipeline	CPU load (%)	RAM load (%)	Frame rate (fps)	Processing Time (ms)	GPU load (%)
Videotestsrc HD	2.374	2.483	29.748	9.587	4.78
Videotestsrc 4K	3.634	5.268	29.752	18.826	13.55
Filesrc HD	2.8	3.052	29.748	9.596	5.45
Filesrc 4K	5.792	7.206	40.741	16.605	16.58
Nvarguscamerasrc	4.573	3.467	29.752	8.741	5.11