GStreamer Pan Tilt Zoom and Rotate Element - Tegra Pipelines

This page provides example GStreamer pipelines to test GstPTZR features on a NVIDIA^®Jetson™ TX1/TX2. These pipelines were tested with JetPack 3.1.

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.

GStreamer pipelines

Same input and output resolution with zoom level 1:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

You can change the zoom level by changing "zoom-level".

Magnification (zoom level > 1):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=2 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Reduction (zoom level < 1)

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=0 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

You can perform translations relative to the center of the input image by setting the "pan-level" and "tilt-level" properties or perform translations relative to the current position with "pan-delta" and "tilt-delta".

Translate right (-1 < pan level < 0):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr pan-level=-1 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Translate left (0 < pan level < 1):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr pan-level=1 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Translate down (-1 < tilt level < 0):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr tilt-level=-1 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Translate up (0 < tilt level < 1):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr tilt-level=100 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

The property "normalize-translation" (true by default) lets you switch the translation normalization ON and OFF. When translation normalization is true the minimum and maximum translation values are mapped to -1 and 1. On the other hand, when it is set to false, the pan and tilt levels reflect pixels on the input image.

Turn off translation normalization:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr normalize-translation=false ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

The property "output-reference" (true by default) lets you change the coordinate system used as reference for translations. This is specially useful after performing a rotation. If you want to move over the input image axis, it must be changed to false.

Set input coordinate system as reference for translations:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr output-reference=false ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

To rotate the input modify the "rotate-level" property (degrees). You can also perform rotation relative to the current angle with "rotate-delta":

Rotate counter-clockwise (rotate level < 0):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr rotate-level=-100 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Rotate clockwise (rotate level > 0):

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr rotate-level=100 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

The element supports input and output of any resolution and aspect ratio

Smaller input image, different aspect ratio:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1024,height=768,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Bigger input image, different aspect ratio:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1280,height=1024,framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

You can use ranges for input and output caps or leave them unspecified

Resolution range:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=[480,1280],height=[680,1024],framerate=30/1'
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink

Unspecified caps:

NV_CAPS='video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1'
IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! capsfilter caps="$NV_CAPS" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr zoom-level=1 ! videoconvert ! xvimagesink

Gstd pipelines

Run the element with gstd to change parameters dynamically:

IN_CAPS='video/x-raw,format=RGBA,width=1920,height=1080,framerate=30/1' 
OUT_CAPS='video/x-raw,format=RGBA,width=1600,height=900,framerate=30/1'
gstd&
gstd-client pipeline_create p0 nvcamerasrc fpsRange="30.0 30.0" ! nvvidconv flip-method=2 ! capsfilter caps="$IN_CAPS" ! ptzr name=ptzr0 ! capsfilter caps="$OUT_CAPS" ! videoconvert ! xvimagesink
gstd-client pipeline_play p0

Change propierties:

gstd-client element_set p0 "ptzr0 zoom-level 0.0" 
gstd-client element_set p0 "ptzr0 zoom-level 2.0" 
gstd-client element_set p0 "ptzr0 zoom-level 1.0" 
gstd-client element_set p0 "ptzr0 tilt-level -1.0" 
gstd-client element_set p0 "ptzr0 tilt-level 1.0" 
gstd-client element_set p0 "ptzr0 tilt-level 0.0" 
gstd-client element_set p0 "ptzr0 pan-level -1.0" 
gstd-client element_set p0 "ptzr0 pan-level 1.0" 
gstd-client element_set p0 "ptzr0 pan-level 0.0" 
gstd-client element_set p0 "ptzr0 tilt-delta -1.0" 
gstd-client element_set p0 "ptzr0 tilt-delta 1.0" 
gstd-client element_set p0 "ptzr0 tilt-delta 0.0" 
gstd-client element_set p0 "ptzr0 pan-delta -1.0" 
gstd-client element_set p0 "ptzr0 pan-delta 1.0" 
gstd-client element_set p0 "ptzr0 pan-delta 0.0" 
gstd-client element_set p0 "ptzr0 rotate-level -90.0" 
gstd-client element_set p0 "ptzr0 rotate-level 90.0" 
gstd-client element_set p0 "ptzr0 rotate-level 0.0" 
gstd-client element_set p0 "ptzr0 rotate-delta -90.0" 
gstd-client element_set p0 "ptzr0 rotate-delta 90.0" 
gstd-client element_set p0 "ptzr0 rotate-delta 0.0" 
gstd-client element_set p0 "ptzr0 normalize-translation false" 
gstd-client element_set p0 "ptzr0 normalize-translation true" 
gstd-client element_set p0 "ptzr0 output-reference false" 
gstd-client element_set p0 "ptzr0 output-reference true"

Stop the pipeline:

gstd-client pipeline_stop p0
gstd-client pipeline_delete p0

Pipelines performance on Jetson TX1

This section presents some performance and latency results obtained with the TX1 running JetPack 3.1. All the pipelines tested obtained 2592x1944@30 buffers from the default camera sensor (OV5693) and transformed them, using ptzr, to a 1920x1080 output at variable frame rates. Even though these pipelines only perform scaling, all ptzr operations should have the same performance. All results are obtained with 10 sample averages.

Pipelines

Without queues, xvimagesink:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM),width=2592,height=1944' ! nvvidconv ! 'video/x-raw,format=RGBA,width=2592,height=1944' ! ptzr ! 'video/x-raw,format=RGBA,width=1920,height=1080' ! videoconvert ! xvimagesink sync=false

With queues, xvimagesink:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM),width=2592,height=1944' ! queue ! nvvidconv ! 'video/x-raw,format=RGBA,width=2592,height=1944' ! queue ! ptzr ! 'video/x-raw,width=1920,height=1080' ! queue ! videoconvert ! xvimagesink sync=false

With queues, xvimagesink and nvvidconv:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM),width=2592,height=1944' ! queue ! nvvidconv ! 'video/x-raw,format=RGBA,width=2592,height=1944' ! queue ! ptzr ! 'video/x-raw,width=1920,height=1080' ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! queue ! nvvidconv ! xvimagesink sync=false

Without queues, nvoverlaysink:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM),width=2592,height=1944' ! nvvidconv ! 'video/x-raw,format=RGBA,width=2592,height=1944' ! videoscale ! 'video/x-raw,format=RGBA,width=1920,height=1080' ! nvvidconv ! nvoverlaysink sync=false

With queues, nvoverlaysink:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM),width=2592,height=1944' ! queue ! nvvidconv ! 'video/x-raw,format=RGBA,width=2592,height=1944' ! queue ! ptzr ! 'video/x-raw,format=RGBA,width=1920,height=1080' ! queue ! nvvidconv ! nvoverlaysink sync=false

Results

Pipeline	ARM load (%)	GPU load (%)	Frame rate (fps)	Latency (ms)
Without queues, xvimagesink	25.3	47.4	15.9	690
With queues, xvimagesink	61.5	41.8	29.8	644
With queues, xvimagesink and nvvidconv	53.7	79.2	29.3	460
Without queues, nvoverlaysink	15.5	53.0	26.3	402
With queues, nvoverlaysink	22.1	54.4	30.1	344

Note

Latency introduced by the ptzr element depends on the input resolution. For 1080p latency associated with the element is around 58ms and for 2592x1944 (this case) is around 116ms.

Previous: iMX8 Pipelines

Index

Next: RB5/RB6 Pipelines

❯