Xilinx ZYNQ UltraScale+ MPSoC Accelerating Image Processing Elements
| Xilinx ZYNQ UltraScale+ MPSoC | ||||
|---|---|---|---|---|
| ||||
| Introduction | ||||
| Getting Started | ||||
| Xilinx Kria | ||||
| Development | ||||
| Development Flows Examples | ||||
| GStreamer | ||||
|
||||
| Contact Us |
Xilinx MPSoC's VCU IP is capable of simultaneously encoding and decoding up to 32 streams of H.264 and H.265 codecs, with a maximum bandwidth equivalent to 4K60. 8K encoding/decoding is also supported at a 15 FPS reduced rate mode.
It is important to notice that to make use of the VCU, firmware with the VCU enabled must be loaded into the FPGA. For this wiki, the smartcam example firmware was used during testing. Xilinx seems to have stopped including these apps in the newer versions of Petalinux, for this reason, you may need to add them manually. The firmware can be found in this 
repository and follow the instructions to add a firmware recipe to the Petalinux build.
Video Encoding
H.264
The H.264 encoder is available through the omxh264enc GStreamer element.
gst-inspect-1.0 omxh264enc
|
|---|
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh264enc
Factory Details:
Rank primary + 1 (257)
Long-name OpenMAX H.264 Video Encoder
Klass Codec/Encoder/Video/Hardware
Description Encode H.264 video streams
Author Sebastian Dr?ge <sebastian.droege@collabora.co.uk>
Plugin Details:
Name omx
Description GStreamer OpenMAX Plug-ins
Filename /usr/lib/gstreamer-1.0/libgstomx.so
Version 1.18.5
License LGPL
Source module gst-omx
Source release date 2021-09-08
Binary package GStreamer OpenMAX Plug-ins
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----GstOMXVideoEnc
+----GstOMXH264Enc
+----GstOMXH264Enc-omxh264enc
Implemented Interfaces:
GstPreset
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw(memory:XLNXLL)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h264
width: [ 16, 4096 ]
height: [ 16, 4096 ]
framerate: [ 0/1, 2147483647/1 ]
stream-format: byte-stream
alignment: { (string)au, (string)nal }
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
aspect-ratio : Display aspect ratio of the video sequence to be written in SPS/VUI
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncAspectRatio" Default: 0, "auto"
(0): auto - 4:3 for SD video,16:9 for HD video,unspecified for unknown format
(1): 1-1 - 1:1 aspect ratio
(2): 4-3 - 4:3 aspect ratio
(3): 16-9 - 16:9 aspect ratio
(4): none - Aspect ratio information is not present in the stream
b-frames : Number of B-frames between two consecutive P-frames
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
constrained-intra-prediction: If enabled, prediction only uses residual data and decoded samples from neighbouring coding blocks coded using intra prediction modes
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
control-rate : Bitrate control method
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncControlRate" Default: -1, "default"
(0): disable - Disable
(1): variable - Variable
(2): constant - Constant
(3): variable-skip-frames - Variable Skip Frames
(4): constant-skip-frames - Constant Skip Frames
(2130706434): capped-variable - Capped Variable
(2130706433): low-latency - Low Latency
(-1): default - Component Default
cpb-size : Coded Picture Buffer as specified in the HRD model in msec. Not used when control-rate=disable
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 3000
default-roi-quality : The default quality level to apply to each Region of Interest
flags: readable, writable
Enum "GstOMXVideoEncRoiQuality" Default: 0, "high"
(0): high - Delta QP of -5
(1): medium - Delta QP of 0
(2): low - Delta QP of +5
(3): dont-care - Maximum delta QP value
(4): intra - Region all LCU encoded with intra prediction mode
dependent-slice : If encoding with multiple slices, specify whether the additional slices are dependent slice segments or regular slices
flags: readable, writable
Boolean. Default: false
entropy-mode : Entropy mode for encoding process
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH264EncEntropyMode" Default: -1, "default"
(0): CAVLC - CAVLC entropy mode
(1): CABAC - CABAC entropy mode
(-1): default - Component Default
filler-data : Enable/Disable Filler Data NAL units for CBR rate control
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: true
gdr-mode : Gradual Decoder Refresh scheme mode. Only used if gop-mode=low-delay-p
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncGdrMode" Default: 0, "disabled"
(0): disabled - No GDR
(1): vertical - Gradual refresh using a vertical bar moving from left to right
(2): horizontal - Gradual refresh using a horizontal bar moving from top to bottom
gop-length : Distance between two consecutive I frames
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 1000 Default: 30
gop-mode : Group Of Pictures mode
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncGopMode" Default: 0, "basic"
(0): basic - Basic GOP settings
(1): basic-b - Basic GOP settings, includes only B-frames
(2): pyramidal - Advanced GOP pattern with hierarchical B-frames
(3): pyramidal-b - Advanced GOP pattern with hierarchical B-frames, includes only B-frames
(5): low-delay-p - Single I-frame followed by P-frames only
(6): low-delay-b - Single I-frame followed by B-frames only
(4): adaptive - Advanced GOP pattern with adaptive B-frames
hlg-sdr-compatible : If enabled and input caps contain HLG colorimetry, insert BT2020 EOTF and ATC SEI instead
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
initial-delay : The initial removal delay as specified in the HRD model in msec. Not used when control-rate=disable
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 1500
input-crop : Cropping parameters for input video buffer ('<left, top, width, height>')
flags: writable
GstValueArray of GValues of type "gint" Write only
interval-intraframes: Interval of coding Intra frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
-h264,alignment=nal ! ...).
flags: readable, writable
Enum "GstOMXVideoEncLatencyMode" Default: 0, "normal"
(0): normal - Normal mode
(1): low-latency - Low latency mode
(-1): default - Component Default
long-term-freq : Periodicity of LongTerm reference picture marking in encoding process Units in frames, distance between two consequtive long-term reference pictures
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
long-term-ref : If enabled, encoder accepts dynamically inserting and using long-term reference picture events from upstream elements
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
look-ahead : The number of frames processed ahead of second pass encoding. If smaller than 2, dual pass encoding is disabled
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
loop-filter-alpha-c0-offset: Alpha C0 offset for the deblocking filter, used only when loop-filter-mode is enabled
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Integer. Range: -6 - 6 Default: -1
loop-filter-beta-offset: Beta offset for the deblocking filter, used only when loop-filter-mode is enabled
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Integer. Range: -6 - 6 Default: -1
loop-filter-mode : Enable or disable the deblocking filter (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH264EncLoopFilter" Default: -1, "default"
(0): enable - Enable deblocking filter
(1): disable - Disable deblocking filter
(2): disable-slice-boundary - Disables deblocking filter on slice boundary
(-1): default - Component Default
low-bandwidth : If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
max-bitrate : Max bitrate in Kbps, only used if control-rate=variable (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
max-consecutive-skip: Max number of consecutive frames to be skipped. Only applicable if skip-frame is enabled
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
ze will not have any effect
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
icture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
flags: writable
GstValueArray of GValues of type "gint" Write only
max-qp : Maximum QP value allowed for the rate control
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 51 Default: 51
le
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
min-qp : Minimum QP value allowed for the rate control
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 51 Default: 10
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "omxh264enc-omxh264enc0"
egularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 1 - 4294967295 Default: 4294967295
output-crop : Cropping parameters for output bitstream ('<left, top, width, height>')
flags: writable
GstValueArray of GValues of type "gint" Write only
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
periodicity-idr : Periodicity of IDR frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
periodicty-idr : Periodicity of IDR frames (0xffffffff=component default) DEPRECATED - only for backwards compat
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
prefetch-buffer : Enable/Disable L2Cache buffer in encoding process
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
qp-mode : QP control mode used by the VCU encoder
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncQpMode" Default: -1, "default"
(0): uniform - Use the same QP for all coding units of the frame
(1): roi - Adjust QP according to the regions of interest defined on each frame. Must be set to handle ROI metadata.
(2): auto - Let the VCU encoder change the QP for each coding unit according to its content
(3): load-qp-absolute - Uses absolute QP values set by user. Must be set to use External QP buffer
(4): load-qp-relative - Uses Relative/Delta QP values set by user. Must be set to use External QP buffer
(-1): default - Component Default
quant-b-frames : Quantization parameter for B-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-i-frames : Quantization parameter for I-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-p-frames : Quantization parameter for P-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
ref-frames : Number of reference frames used for inter-motion search (0=component default)
flags: readable, writable, changeable only in NULL or READY state
Unknown type 16 "guchar"
scaling-list : Scaling list mode
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncScalingList" Default: 1, "default"
(1): default - Default scaling list mode
(0): flat - Flat scaling list mode
Skip. Only used if control-rate=constant or variable and b-frames are less than 2
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
slice-size : Target slice size (in bytes) that the encoder uses to automatically split the bitstream into approximately equally-sized slices
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 65535 Default: 0
target-bitrate : Target bitrate in Kbps (0xffffffff=component default)
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
uniform-slice-type : Enable/Disable uniform slice type in slice header
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
use-out-port-pool : Use a buffer pool on the output port
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
cture size will be limited to max-picture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
flags: writable
GstValueArray of GValues of type "gint" Write only
y444-to-gray : Enable/Disable YUV format manipulation to Y8
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
|
Performance
Using AR1335 camera through AP1302 ISP at 1080p:
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=1920, height=1080, format=NV12, framerate=30/1 ! omxh264enc ! perf ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 2 % | 30.000 | Not tested |
Using AR1335 camera through AP1302 ISP at 4K:
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! omxh264enc ! perf ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 3 % | 30.000 | Not tested |
H.265
The H.265 encoder is available through the omxh265enc GStreamer element.
gst-inspect-1.0 omxh265enc
|
|---|
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh265enc
Factory Details:
Rank primary + 1 (257)
Long-name OpenMAX H.265 Video Encoder
Klass Codec/Encoder/Video/Hardware
Description Encode H.265 video streams
Author Sebastian Dr?ge <sebastian.droege@collabora.co.uk>
Plugin Details:
Name omx
Description GStreamer OpenMAX Plug-ins
Filename /usr/lib/gstreamer-1.0/libgstomx.so
Version 1.18.5
License LGPL
Source module gst-omx
Source release date 2021-09-08
Binary package GStreamer OpenMAX Plug-ins
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----GstOMXVideoEnc
+----GstOMXH265Enc
+----GstOMXH265Enc-omxh265enc
Implemented Interfaces:
GstPreset
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw(format:Interlaced)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
interlace-mode: alternate
video/x-raw(memory:XLNXLL)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h265
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
stream-format: byte-stream
aligmment: { (string)au, (string)nal }
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
aspect-ratio : Display aspect ratio of the video sequence to be written in SPS/VUI
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncAspectRatio" Default: 0, "auto"
(0): auto - 4:3 for SD video,16:9 for HD video,unspecified for unknown format
(1): 1-1 - 1:1 aspect ratio
(2): 4-3 - 4:3 aspect ratio
(3): 16-9 - 16:9 aspect ratio
(4): none - Aspect ratio information is not present in the stream
b-frames : Number of B-frames between two consecutive P-frames
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
constrained-intra-prediction: If enabled, prediction only uses residual data and decoded samples from neighbouring coding blocks coded using intra prediction modes
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
control-rate : Bitrate control method
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncControlRate" Default: -1, "default"
(0): disable - Disable
(1): variable - Variable
(2): constant - Constant
(3): variable-skip-frames - Variable Skip Frames
(4): constant-skip-frames - Constant Skip Frames
(2130706434): capped-variable - Capped Variable
(2130706433): low-latency - Low Latency
(-1): default - Component Default
cpb-size : Coded Picture Buffer as specified in the HRD model in msec. Not used when control-rate=disable
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 3000
default-roi-quality : The default quality level to apply to each Region of Interest
flags: readable, writable
Enum "GstOMXVideoEncRoiQuality" Default: 0, "high"
(0): high - Delta QP of -5
(1): medium - Delta QP of 0
(2): low - Delta QP of +5
(3): dont-care - Maximum delta QP value
(4): intra - Region all LCU encoded with intra prediction mode
dependent-slice : If encoding with multiple slices, specify whether the additional slices are dependent slice segments or regular slices
flags: readable, writable
Boolean. Default: false
filler-data : Enable/Disable Filler Data NAL units for CBR rate control
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: true
gdr-mode : Gradual Decoder Refresh scheme mode. Only used if gop-mode=low-delay-p
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncGdrMode" Default: 0, "disabled"
(0): disabled - No GDR
(1): vertical - Gradual refresh using a vertical bar moving from left to right
(2): horizontal - Gradual refresh using a horizontal bar moving from top to bottom
gop-length : Distance between two consecutive I frames
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 1000 Default: 30
gop-mode : Group Of Pictures mode
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncGopMode" Default: 0, "basic"
(0): basic - Basic GOP settings
(1): basic-b - Basic GOP settings, includes only B-frames
(2): pyramidal - Advanced GOP pattern with hierarchical B-frames
(3): pyramidal-b - Advanced GOP pattern with hierarchical B-frames, includes only B-frames
(5): low-delay-p - Single I-frame followed by P-frames only
(6): low-delay-b - Single I-frame followed by B-frames only
(4): adaptive - Advanced GOP pattern with adaptive B-frames
hlg-sdr-compatible : If enabled and input caps contain HLG colorimetry, insert BT2020 EOTF and ATC SEI instead
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
initial-delay : The initial removal delay as specified in the HRD model in msec. Not used when control-rate=disable
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 1500
input-crop : Cropping parameters for input video buffer ('<left, top, width, height>')
flags: writable
GstValueArray of GValues of type "gint" Write only
interval-intraframes: Interval of coding Intra frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
-h264,alignment=nal ! ...).
flags: readable, writable
Enum "GstOMXVideoEncLatencyMode" Default: 0, "normal"
(0): normal - Normal mode
(1): low-latency - Low latency mode
(-1): default - Component Default
long-term-freq : Periodicity of LongTerm reference picture marking in encoding process Units in frames, distance between two consequtive long-term reference pictures
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
long-term-ref : If enabled, encoder accepts dynamically inserting and using long-term reference picture events from upstream elements
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
look-ahead : The number of frames processed ahead of second pass encoding. If smaller than 2, dual pass encoding is disabled
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
loop-filter-beta-offset: Beta offset for the deblocking filter, used only when loop-filter-mode is enabled
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Integer. Range: -6 - 6 Default: -1
loop-filter-mode : Enable or disable the deblocking filter (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH265EncLoopFilter" Default: -1, "default"
(0): enable - Enable deblocking filter
(1): disable - Disable deblocking filter
(2): disable-cross-slice - Disable deblocking filter on slice boundary
(3): disable-cross-tile - Disable deblocking filter on tile boundary
(4): disable-slice-and-tile - Disable deblocking filter on slice and tile boundary
(-1): default - Component Default
loop-filter-tc-offset: TC offset for the deblocking filter, used only when loop-filter-mode is enabled
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Integer. Range: -6 - 6 Default: -1
low-bandwidth : If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
max-bitrate : Max bitrate in Kbps, only used if control-rate=variable (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
max-consecutive-skip: Max number of consecutive frames to be skipped. Only applicable if skip-frame is enabled
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
ze will not have any effect
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 0
icture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
flags: writable
GstValueArray of GValues of type "gint" Write only
max-qp : Maximum QP value allowed for the rate control
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 51 Default: 51
le
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
min-qp : Minimum QP value allowed for the rate control
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 51 Default: 10
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "omxh265enc-omxh265enc0"
egularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 1 - 4294967295 Default: 4294967295
output-crop : Cropping parameters for output bitstream ('<left, top, width, height>')
flags: writable
GstValueArray of GValues of type "gint" Write only
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
periodicity-idr : Periodicity of IDR frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
prefetch-buffer : Enable/Disable L2Cache buffer in encoding process
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
qp-mode : QP control mode used by the VCU encoder
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncQpMode" Default: -1, "default"
(0): uniform - Use the same QP for all coding units of the frame
(1): roi - Adjust QP according to the regions of interest defined on each frame. Must be set to handle ROI metadata.
(2): auto - Let the VCU encoder change the QP for each coding unit according to its content
(3): load-qp-absolute - Uses absolute QP values set by user. Must be set to use External QP buffer
(4): load-qp-relative - Uses Relative/Delta QP values set by user. Must be set to use External QP buffer
(-1): default - Component Default
quant-b-frames : Quantization parameter for B-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-i-frames : Quantization parameter for I-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-p-frames : Quantization parameter for P-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
scaling-list : Scaling list mode
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncScalingList" Default: 1, "default"
(1): default - Default scaling list mode
(0): flat - Flat scaling list mode
Skip. Only used if control-rate=constant or variable and b-frames are less than 2
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
slice-size : Target slice size (in bytes) that the encoder uses to automatically split the bitstream into approximately equally-sized slices
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 65535 Default: 0
target-bitrate : Target bitrate in Kbps (0xffffffff=component default)
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
uniform-slice-type : Enable/Disable uniform slice type in slice header
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
use-out-port-pool : Use a buffer pool on the output port
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
cture size will be limited to max-picture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
flags: writable
GstValueArray of GValues of type "gint" Write only
y444-to-gray : Enable/Disable YUV format manipulation to Y8
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
|
Performance
Using AR1335 camera through AP1302 ISP at 1080p:
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=1920, height=1080, format=NV12, framerate=30/1 ! omxh265enc ! perf ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 2 % | 30.000 | Not tested |
Using AR1335 camera through AP1302 ISP at 4K:
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! omxh265enc ! perf ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 3 % | 30.000 | Not tested |
Video Decoding
H.264
The H.264 decoder is available through the omxh264dec GStreamer element.
gst-inspect-1.0 omxh264dec
|
|---|
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh264dec
Factory Details:
Rank primary + 1 (257)
Long-name OpenMAX H.264 Video Decoder
Klass Codec/Decoder/Video/Hardware
Description Decode H.264 video streams
Author Sebastian Dr?ge <sebastian.droege@collabora.co.uk>
Plugin Details:
Name omx
Description GStreamer OpenMAX Plug-ins
Filename /usr/lib/gstreamer-1.0/libgstomx.so
Version 1.18.5
License LGPL
Source module gst-omx
Source release date 2021-09-08
Binary package GStreamer OpenMAX Plug-ins
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstOMXVideoDec
+----GstOMXH264Dec
+----GstOMXH264Dec-omxh264dec
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h264
alignment: au
stream-format: byte-stream
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
video/x-h264
alignment: nal
stream-format: byte-stream
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw(memory:GLMemory)
format: RGBA
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw(memory:XLNXLL, format:Interlaced)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
interlace-mode: alternate
video/x-raw(format:Interlaced)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
interlace-mode: alternate
video/x-raw(memory:XLNXLL)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
g high bitrate streams. Decreasing it reduces the memory footprint
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 2 - 16 Default: 5
t to 'au' for normal latency and 'nal' for subframe latency (e.g. ... ! video/x-h264,alignment=nal ! omxh264dec low-latency=1 ! ...).
flags: readable, writable
Enum "GstOMXVideoDecLatencyMode" Default: 0, "normal"
(0): normal - Normal mode
(1): reduced-latency - Low ref dpb mode(reduced-latency)
(2): low-latency - Low latency mode
(-1): default - Component Default
low-latency : When enabled, buffers will be pushed before they are evicted from the DBP, reducing decoding latency.
flags: readable, writable
Boolean. Default: false
max-errors : Max consecutive decoder errors before returning flow error
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: 10
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "omxh264dec-omxh264dec0"
output-position : Position coordinates of start of active pixel data in video buffer ('<position_x, position_y>')
flags: writable
GstValueArray of GValues of type "gint" Write only
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: true
buffer and process them.
flags: readable, writable
Boolean. Default: false
|
Performance
1080p
File: 1080p-h264-60fps.mp4
gst-launch-1.0 filesrc location="1080p-h264-60fps.mp4" ! qtdemux ! h264parse ! queue ! omxh264dec ! perf ! queue ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 105 % | 40.582 | Not tested |
4K
File:
gst-launch-1.0 filesrc location="" ! qtdemux ! queue ! omxh264dec ! perf ! queue ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | Not tested | Not tested | Not tested |
H.265
The H.265 decoder is available through the omxh265dec GStreamer element.
gst-inspect-1.0 omxh265dec
|
|---|
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh265dec
Rank primary + 1 (257)
Long-name OpenMAX H.265 Video Decoder
Klass Codec/Decoder/Video/Hardware
Description Decode H.265 video streams
Author Sebastian Dr?ge <sebastian.droege@collabora.co.uk>
Plugin Details:
Name omx
Description GStreamer OpenMAX Plug-ins
Filename /usr/lib/gstreamer-1.0/libgstomx.so
Version 1.18.5
License LGPL
Source module gst-omx
Source release date 2021-09-08
Binary package GStreamer OpenMAX Plug-ins
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstOMXVideoDec
+----GstOMXH265Dec
+----GstOMXH265Dec-omxh265dec
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h265
alignment: au
stream-format: byte-stream
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
video/x-h265
alignment: nal
stream-format: byte-stream
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw(memory:GLMemory)
format: RGBA
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw(memory:XLNXLL, format:Interlaced)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
interlace-mode: alternate
video/x-raw(format:Interlaced)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
interlace-mode: alternate
video/x-raw(memory:XLNXLL)
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
g high bitrate streams. Decreasing it reduces the memory footprint
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 2 - 16 Default: 5
t to 'au' for normal latency and 'nal' for subframe latency (e.g. ... ! video/x-h264,alignment=nal ! omxh264dec low-latency=1 ! ...).
flags: readable, writable
Enum "GstOMXVideoDecLatencyMode" Default: 0, "normal"
(0): normal - Normal mode
(1): reduced-latency - Low ref dpb mode(reduced-latency)
(2): low-latency - Low latency mode
(-1): default - Component Default
low-latency : When enabled, buffers will be pushed before they are evicted from the DBP, reducing decoding latency.
flags: readable, writable
Boolean. Default: false
max-errors : Max consecutive decoder errors before returning flow error
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: 10
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "omxh265dec-omxh265dec0"
output-position : Position coordinates of start of active pixel data in video buffer ('<position_x, position_y>')
flags: writable
GstValueArray of GValues of type "gint" Write only
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: true
buffer and process them.
flags: readable, writable
Boolean. Default: false
|
Performance
1080p
File: 1080p-hevc-30fps.mkv
gst-launch-1.0 filesrc location="1080p-hevc-30fps.mkv" ! matroskademux ! h265parse ! queue ! omxh265dec ! perf ! queue ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 108 % | 40.109 | Not tested |
4K
File: 4k-hevc-60fps.mkv
gst-launch-1.0 filesrc location="4k-hevc-60fps.mkv" ! matroskademux ! h265parse ! queue ! omxh265dec ! perf ! queue ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 103 % | 7.919 | Not tested |
Simultaneous encoding and decoding
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! omxh265enc ! queue ! omxh265dec ! perf ! queue ! fakesink
| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| KV260 | 102 % | 9.775 | Not tested |