View source for JetsonTX2/GStreamer/H264

<noinclude>
{{JetsonTX2/Head|next=GStreamer/H265|previous=GStreamer/Capture and Display Pipelines}}
</noinclude>

<table>
<tr>
<td><div class="clear; float:right">__TOC__</div></td>
<td valign=top>
{{GStreamer debug}}
</td>
</table>

== H264 encoding ==

=== videotestsrc ===

ARM load: 157% but only videotestsrc consumes 100%. According to the NVIDIA forum when using videotestsrc the system is also doing memory copies to feed the encoder with NVMM memory buffers so that loads more the ARM.

<pre>
gst-launch-1.0 videotestsrc num-buffers=150 ! 'video/x-raw, format=(string)I420,width=(int)640, height=(int)480' ! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=test.h264 -e
</pre>

VLC can play the file correctly at 30fps and reports as codec: H264-MPEG-4 AVC (part 10) (h264). The fan on top of the SoC didn't turn on.

=== nvcamerasrc ===

<pre>
gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxh264enc bitrate=8000000 \
! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=test.h264 -e
</pre>

Codecvisa reports bitrate on 7.96Mbps when variable mode is selected on the encoder.

ARM consumption 45.4% of one core (4 cores available 400% - this means something like 16.75% of total). However, only the capture is 11.25% so the h264 consumes '''5.5%''' 

<pre>
RAM 1279/7853MB (lfb 1237x4MB) cpu [0%@1267,off,off,0%@1267,0%@1267,0%@1267]
RAM 1280/7853MB (lfb 1237x4MB) cpu [8%@345,off,off,19%@345,10%@345,12%@345]
RAM 1279/7853MB (lfb 1237x4MB) cpu [16%@345,off,off,16%@345,7%@345,13%@345]
RAM 1279/7853MB (lfb 1237x4MB) cpu [18%@345,off,off,11%@345,10%@345,16%@345]
RAM 1279/7853MB (lfb 1236x4MB) cpu [24%@345,off,off,12%@345,11%@345,13%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [15%@345,off,off,14%@345,10%@345,14%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [18%@345,off,off,14%@345,16%@345,11%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [15%@345,off,off,19%@345,13%@345,16%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [14%@345,off,off,18%@345,15%@345,11%@345]
RAM 1280/7853MB (lfb 1236x4MB) cpu [20%@345,off,off,13%@345,13%@345,14%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [15%@345,off,off,14%@345,18%@345,12%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [15%@345,off,off,17%@345,10%@345,13%@345]
RAM 1279/7853MB (lfb 1235x4MB) cpu [18%@345,off,off,19%@345,6%@345,13%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [15%@345,off,off,19%@345,13%@345,10%@345]
RAM 1280/7853MB (lfb 1235x4MB) cpu [17%@345,off,off,18%@345,10%@345,14%@345]
RAM 1235/7853MB (lfb 1235x4MB) cpu [11%@345,off,off,7%@345,7%@345,8%@345]
RAM 1234/7853MB (lfb 1235x4MB) cpu [0%@345,off,off,1%@345,0%@345,0%@345]
RAM 1234/7853MB (lfb 1235x4MB) cpu [0%@345,off,off,3%@345,0%@345,0%@345]

</pre>

<pre>
 1574 root      20   0  9.979g 214836  44684 S  45.4  2.7   7:10.27 nvcamera-daemon                                                           
11614 nvidia    20   0  700192  12800   9236 S   8.6  0.2   0:00.88 gst-launch-1.0           
</pre>

The encoder exposes the following parameters

<pre>
  control-rate        : Bitrate control method
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncControlRate" Default: 1, "variable"
                           (0): disable          - Disable
                           (1): variable         - Variable
                           (2): constant         - Constant
                           (3): variable-skip-frames - Variable Skip Frames
                           (4): constant-skip-frames - Constant Skip Frames
  bitrate             : Target bitrate
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4000000 
  quant-i-frames      : Quantization parameter for I-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-p-frames      : Quantization parameter for P-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-b-frames      : Quantization parameter for B-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  iframeinterval      : Encoding Intra Frame occurance frequency
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  SliceIntraRefreshEnable: Enable Slice Intra Refresh while encoding
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  SliceIntraRefreshInterval: Set SliceIntraRefreshInterval
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 60 
  bit-packetization   : Whether or not Packet size is based upon Number Of bits
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  vbv-size            : virtual buffer size = vbv-size * (bitrate/fps)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 30 Default: 10 
  temporal-tradeoff   : Temporal Tradeoff value for encoder
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOmxVideoEncTemporalTradeoffType" Default: 0, "Do not drop frames"
                           (0): Do not drop frames - GST_OMX_VIDENC_DROP_NO_FRAMES
                           (1): Drop 1 in 5 frames - GST_OMX_VIDENC_DROP_1_IN_5_FRAMES
                           (2): Drop 1 in 3 frames - GST_OMX_VIDENC_DROP_1_IN_3_FRAMES
                           (3): Drop 1 in 2 frames - GST_OMX_VIDENC_DROP_1_IN_2_FRAMES
                           (4): Drop 2 in 3 frames - GST_OMX_VIDENC_DROP_2_IN_3_FRAMES
  EnableMVBufferMeta  : Enable Motion Vector Meta data for encoding
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  qp-range            : Qunatization range for P and I frame,
			 Use string with values of Qunatization Range 
			 in MinQpP-MaxQpP:MinQpI-MaxQpP:MinQpB-MaxQpB order, to set the property.
                        flags: readable, writable
                        String. Default: "-1,-1:-1,-1:-1,-1"
  MeasureEncoderLatency: Enable Measure Encoder latency Per Frame
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  EnableTwopassCBR    : Enable two pass CBR while encoding
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  preset-level        : HW preset level for encoder
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncHwPreset" Default: 0, "UltraFastPreset"
                           (0): UltraFastPreset  - UltraFastPreset for high perf
                           (1): FastPreset       - FastPreset
                           (2): MediumPreset     - MediumPreset
                           (3): SlowPreset       - SlowPreset
  insert-sps-pps      : Insert H.264 SPS, PPS at every IDR frame
                        flags: readable, writable
                        Boolean. Default: false
  num-B-Frames        : Number of B Frames between two reference frames (not recommended)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 2 Default: 0 
  slice-header-spacing: Slice Header Spacing number of macroblocks/bits in one packet
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Long. Range: 0 - 18446744073709551615 Default: 0 
  profile             : Set profile for encode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOmxVideoEncProfileType" Default: 1, "baseline"
                           (1): baseline         - GST_OMX_VIDENC_BASELINE_PROFILE
                           (2): main             - GST_OMX_VIDENC_MAIN_PROFILE
                           (8): high             - GST_OMX_VIDENC_HIGH_PROFILE


</pre>

== Dual H264 encoding ==

<pre>
gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! tee name=streams \
streams. ! omxh264enc bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=testA.h264 \
streams. ! omxh264enc bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=testB.h264
</pre>


ARM consumption ~44.1% of one core (4 cores available 400% - this means something like 16.5% of total). However, only the capture is 11.25% so the dual h264 consumes '''5.25%'''

<pre>
RAM 1320/7853MB (lfb 1221x4MB) cpu [0%@1420,off,off,0%@1420,0%@1420,0%@1420]
RAM 1320/7853MB (lfb 1221x4MB) cpu [19%@345,off,off,16%@345,10%@345,9%@345]
RAM 1320/7853MB (lfb 1221x4MB) cpu [18%@345,off,off,18%@345,16%@345,13%@345]
RAM 1321/7853MB (lfb 1221x4MB) cpu [20%@345,off,off,16%@345,19%@345,13%@345]
RAM 1321/7853MB (lfb 1221x4MB) cpu [19%@345,off,off,14%@345,12%@345,15%@345]
RAM 1321/7853MB (lfb 1220x4MB) cpu [18%@345,off,off,17%@345,14%@345,13%@345]
RAM 1321/7853MB (lfb 1219x4MB) cpu [19%@345,off,off,12%@345,18%@345,15%@345]
RAM 1321/7853MB (lfb 1219x4MB) cpu [14%@345,off,off,13%@345,14%@345,20%@345]
RAM 1321/7853MB (lfb 1218x4MB) cpu [14%@345,off,off,15%@345,21%@345,18%@345]
RAM 1321/7853MB (lfb 1218x4MB) cpu [15%@345,off,off,20%@345,15%@345,16%@345]
RAM 1321/7853MB (lfb 1217x4MB) cpu [14%@345,off,off,17%@345,11%@345,21%@345]
RAM 1321/7853MB (lfb 1217x4MB) cpu [18%@345,off,off,22%@345,13%@345,18%@345]
RAM 1321/7853MB (lfb 1217x4MB) cpu [15%@345,off,off,19%@345,20%@345,12%@345]
RAM 1321/7853MB (lfb 1216x4MB) cpu [15%@345,off,off,21%@345,16%@345,13%@345]
RAM 1321/7853MB (lfb 1216x4MB) cpu [16%@345,off,off,22%@345,18%@345,12%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [15%@345,off,off,11%@345,11%@345,13%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [1%@345,off,off,2%@345,0%@345,0%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [0%@345,off,off,2%@345,0%@345,0%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [0%@345,off,off,2%@345,0%@345,0%@345]
RAM 1275/7853MB (lfb 1216x4MB) cpu [0%@345,off,off,1%@345,0%@345,0%@345]

</pre>

== H264 UDP Streaming ==

server:
<pre>
CLIENT_IP=<IP_ADDRESS>
gst-launch-1.0 nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 \
! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false
</pre>

PC client:
<pre>
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! xvimagesink sync=false async=false -e
</pre>
 
Using VLC 

*Server: 

<pre>
CLIENT_IP=<IP_ADDRESS>
gst-launch-1.0 nvcamerasrc fpsRange="30 30" intent=3 ! nvvidconv flip-method=6 \
! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
omxh264enc control-rate=2 bitrate=4000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false
</pre>

*Client:
<pre>
vlc -vv --demux h264 --h264-fps 60  udp://@:5000 
</pre>
=== h264 parse missing===
In case gstreamer complains when running the pipeline for missing the h264parse plugin, run the following command to install it: 
<pre>
sudo apt-get install gstreamer1.0-plugins-bad
</pre>

=== Latency ===

*1080p30
*H264
*Bitrate 4Mbps
*Latency=138ms

<noinclude>
{{JetsonTX2/Foot|GStreamer/Capture and Display Pipelines|GStreamer/H265}}
</noinclude>