Jetson TX2 - GStreamer - H264

From RidgeRun Developer Wiki


Previous: GStreamer/Capture and Display Pipelines Index Next: GStreamer/H265


NVIDIA®JETSON TX2™
TX2
Introduction
Getting Started
Compiling Jetson TX2 source code
NVIDIA TX2 NVP Model
GStreamer
Compile GStreamer on Jetson TX1 and TX2
Example pipelines

Capture and Display Pipelines
H264
H265
VP8

nvgstcapture-1.0
Example pipelines for Jetpack 4.2+

Capture and Display
Encoding
Decoding

Streaming
Performance
Jetson TX2 video latency
V4L2 driver support

V4L2 driver for camera sensor or capture chip

V4L2 drivers available for Jetson SoCs
Supported camera sensors

GMSL MAX96705/MAX9286 Sony IMX219 OmniVision OV5647 Omnivision OS08A10 ON Semiconductor AR0231 ON Semiconductor AR1335 Toshiba TC358743

Toshiba TC358840
HowTo's

How to enable A/B redundancy in TX2 Setting up OpenPose on TX2

OpenCV face detection for Jetson TX2
Contact Us
Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.



H264 encoding

videotestsrc

ARM load: 157% but only videotestsrc consumes 100%. According to the NVIDIA forum when using videotestsrc the system is also doing memory copies to feed the encoder with NVMM memory buffers so that loads more the ARM. 

gst-launch-1.0 videotestsrc num-buffers=150 ! 'video/x-raw, format=(string)I420,width=(int)640, height=(int)480' ! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=test.h264 -e

VLC can play the file correctly at 30fps and reports as codec: H264-MPEG-4 AVC (part 10) (h264). The fan on top of the SoC didn't turn on.

nvcamerasrc

gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxh264enc bitrate=8000000 \
! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=test.h264 -e

Codecvisa reports bitrate on 7.96Mbps when variable mode is selected on the encoder.

ARM consumption 45.4% of one core (4 cores available 400% - this means something like 16.75% of total). However, only the capture is 11.25% so the h264 consumes 5.5% 

RAM 1279/7853MB (lfb 1237x4MB) cpu [0%@1267,off,off,0%@1267,0%@1267,0%@1267]
RAM 1280/7853MB (lfb 1237x4MB) cpu [8%@345,off,off,19%@345,10%@345,12%@345]
RAM 1279/7853MB (lfb 1237x4MB) cpu [16%@345,off,off,16%@345,7%@345,13%@345]
RAM 1279/7853MB (lfb 1237x4MB) cpu [18%@345,off,off,11%@345,10%@345,16%@345]
RAM 1279/7853MB (lfb 1236x4MB) cpu [24%@345,off,off,12%@345,11%@345,13%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [15%@345,off,off,14%@345,10%@345,14%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [18%@345,off,off,14%@345,16%@345,11%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [15%@345,off,off,19%@345,13%@345,16%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [14%@345,off,off,18%@345,15%@345,11%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [20%@345,off,off,13%@345,13%@345,14%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [15%@345,off,off,14%@345,18%@345,12%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [15%@345,off,off,17%@345,10%@345,13%@345]
RAM 1279/7853MB (lfb 1235x4MB) cpu [18%@345,off,off,19%@345,6%@345,13%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [15%@345,off,off,19%@345,13%@345,10%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [17%@345,off,off,18%@345,10%@345,14%@345]
RAM 1235/7853MB (lfb 1235x4MB) cpu [11%@345,off,off,7%@345,7%@345,8%@345]
RAM 1234/7853MB (lfb 1235x4MB) cpu [0%@345,off,off,1%@345,0%@345,0%@345]
RAM 1234/7853MB (lfb 1235x4MB) cpu [0%@345,off,off,3%@345,0%@345,0%@345]


 1574 root      20   0  9.979g 214836  44684 S  45.4  2.7   7:10.27 nvcamera-daemon                                                           
11614 nvidia    20   0  700192  12800   9236 S   8.6  0.2   0:00.88 gst-launch-1.0

The encoder exposes the following parameters 

  control-rate        : Bitrate control method
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncControlRate" Default: 1, "variable"
                           (0): disable          - Disable
                           (1): variable         - Variable
                           (2): constant         - Constant
                           (3): variable-skip-frames - Variable Skip Frames
                           (4): constant-skip-frames - Constant Skip Frames
  bitrate             : Target bitrate
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4000000 
  quant-i-frames      : Quantization parameter for I-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-p-frames      : Quantization parameter for P-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-b-frames      : Quantization parameter for B-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  iframeinterval      : Encoding Intra Frame occurance frequency
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  SliceIntraRefreshEnable: Enable Slice Intra Refresh while encoding
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  SliceIntraRefreshInterval: Set SliceIntraRefreshInterval
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 60 
  bit-packetization   : Whether or not Packet size is based upon Number Of bits
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  vbv-size            : virtual buffer size = vbv-size * (bitrate/fps)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 30 Default: 10 
  temporal-tradeoff   : Temporal Tradeoff value for encoder
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOmxVideoEncTemporalTradeoffType" Default: 0, "Do not drop frames"
                           (0): Do not drop frames - GST_OMX_VIDENC_DROP_NO_FRAMES
                           (1): Drop 1 in 5 frames - GST_OMX_VIDENC_DROP_1_IN_5_FRAMES
                           (2): Drop 1 in 3 frames - GST_OMX_VIDENC_DROP_1_IN_3_FRAMES
                           (3): Drop 1 in 2 frames - GST_OMX_VIDENC_DROP_1_IN_2_FRAMES
                           (4): Drop 2 in 3 frames - GST_OMX_VIDENC_DROP_2_IN_3_FRAMES
  EnableMVBufferMeta  : Enable Motion Vector Meta data for encoding
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  qp-range            : Qunatization range for P and I frame,
			 Use string with values of Qunatization Range 
			 in MinQpP-MaxQpP:MinQpI-MaxQpP:MinQpB-MaxQpB order, to set the property.
                        flags: readable, writable
                        String. Default: "-1,-1:-1,-1:-1,-1"
  MeasureEncoderLatency: Enable Measure Encoder latency Per Frame
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  EnableTwopassCBR    : Enable two pass CBR while encoding
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  preset-level        : HW preset level for encoder
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncHwPreset" Default: 0, "UltraFastPreset"
                           (0): UltraFastPreset  - UltraFastPreset for high perf
                           (1): FastPreset       - FastPreset
                           (2): MediumPreset     - MediumPreset
                           (3): SlowPreset       - SlowPreset
  insert-sps-pps      : Insert H.264 SPS, PPS at every IDR frame
                        flags: readable, writable
                        Boolean. Default: false
  num-B-Frames        : Number of B Frames between two reference frames (not recommended)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 2 Default: 0 
  slice-header-spacing: Slice Header Spacing number of macroblocks/bits in one packet
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Long. Range: 0 - 18446744073709551615 Default: 0 
  profile             : Set profile for encode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOmxVideoEncProfileType" Default: 1, "baseline"
                           (1): baseline         - GST_OMX_VIDENC_BASELINE_PROFILE
                           (2): main             - GST_OMX_VIDENC_MAIN_PROFILE
                           (8): high             - GST_OMX_VIDENC_HIGH_PROFILE

Dual H264 encoding

gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! tee name=streams \
streams. ! omxh264enc bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=testA.h264 \
streams. ! omxh264enc bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=testB.h264


ARM consumption ~44.1% of one core (4 cores available 400% - this means something like 16.5% of total). However, only the capture is 11.25% so the dual h264 consumes 5.25% 

RAM 1320/7853MB (lfb 1221x4MB) cpu [0%@1420,off,off,0%@1420,0%@1420,0%@1420]
RAM 1320/7853MB (lfb 1221x4MB) cpu [19%@345,off,off,16%@345,10%@345,9%@345]
RAM 1320/7853MB (lfb 1221x4MB) cpu [18%@345,off,off,18%@345,16%@345,13%@345]
RAM 1321/7853MB (lfb 1221x4MB) cpu [20%@345,off,off,16%@345,19%@345,13%@345]
RAM 1321/7853MB (lfb 1221x4MB) cpu [19%@345,off,off,14%@345,12%@345,15%@345]
RAM 1321/7853MB (lfb 1220x4MB) cpu [18%@345,off,off,17%@345,14%@345,13%@345]
RAM 1321/7853MB (lfb 1219x4MB) cpu [19%@345,off,off,12%@345,18%@345,15%@345]
RAM 1321/7853MB (lfb 1219x4MB) cpu [14%@345,off,off,13%@345,14%@345,20%@345]
RAM 1321/7853MB (lfb 1218x4MB) cpu [14%@345,off,off,15%@345,21%@345,18%@345]
RAM 1321/7853MB (lfb 1218x4MB) cpu [15%@345,off,off,20%@345,15%@345,16%@345]
RAM 1321/7853MB (lfb 1217x4MB) cpu [14%@345,off,off,17%@345,11%@345,21%@345]
RAM 1321/7853MB (lfb 1217x4MB) cpu [18%@345,off,off,22%@345,13%@345,18%@345]
RAM 1321/7853MB (lfb 1217x4MB) cpu [15%@345,off,off,19%@345,20%@345,12%@345]
RAM 1321/7853MB (lfb 1216x4MB) cpu [15%@345,off,off,21%@345,16%@345,13%@345]
RAM 1321/7853MB (lfb 1216x4MB) cpu [16%@345,off,off,22%@345,18%@345,12%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [15%@345,off,off,11%@345,11%@345,13%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [1%@345,off,off,2%@345,0%@345,0%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [0%@345,off,off,2%@345,0%@345,0%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [0%@345,off,off,2%@345,0%@345,0%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [0%@345,off,off,1%@345,0%@345,0%@345]

H264 UDP Streaming

server: 

CLIENT_IP=<IP_ADDRESS>
gst-launch-1.0 nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 \
! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false

PC client: 

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! xvimagesink sync=false async=false -e

Using VLC

  • Server:
CLIENT_IP=<IP_ADDRESS>
gst-launch-1.0 nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 \
! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false
  • Client:
vlc -vv --demux h264 --h264-fps 60  udp://@:5000

h264 parse missing

In case gstreamer complains when running the pipeline for missing the h264parse plugin, run the following command to install it: 

sudo apt-get install gstreamer1.0-plugins-bad

Latency

  • 1080p30
  • H264
  • Bitrate 4Mbps
  • Latency=138ms


Previous: GStreamer/Capture and Display Pipelines Index Next: GStreamer/H265



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=JetsonTX2/GStreamer/H264&oldid=61661"