GStreamer Encoding Latency in NVIDIA Jetson Platforms: Difference between revisions

This wiki is intended to evaluate the latency or processing time of the GStreamer hardware-accelerated encoders available in Jetson platforms.

The results shown here were obtained using a Jetson TX2 platform. The evaluation was intended for optimizing Jetpack 3.3.3 but some results using Jetpack 4.5.1 are included to compare the improvements in new releases.

This evaluation involves two pipelines encoding simultaneously, using H264 codec. The test cases are for the following resolutions:

For this analysis we are only interested in the encoders processing time, therefore the plots will only show the behavior of these.

<div><ul>  

|}

In order to validate if the encoder profile affects the latency we tested the 3 different profiles with the 1080p resolution:

|}

Another common question is if the encoder bitrate affects the latency, so to test it we tried 4 different values with the 1080p resolution:

Note that none of the values seem to make a significant difference, so we can say that the bitrate does not affect the encoder processing time.

<div><ul>  

But it also does not seem to have any effect on latency.

The H265 codec is being used more and more lately, so it is important to validate its performance too:

|}

The idea of these tests is to verify if there were improvements on the encoding with new releases.

<div><ul>  

As we can see, there were some small improvements in the 4.5 release compared to 3.3.

As per the H265 codec in this newer release the results are the following:

NVIDIA has reported in several posts that OMX encoders are deprecated, therefore they recommend to use the V4L2 encoders '''nvv4l2h264enc''' and '''nvv4l2h265enc''' in newer releases. The base pipelines for these are:

</source>

<div><ul>  

As we can see, they have pretty much the same performance than the OMX encoder.

The V4L2 encoders have a property named '''maxperf-enable''' that can be set for decreasing the processing time and improving the performance:

Note that here we have the best time compared to the other encoders and tests, so the '''maxperf-enable''' property seems to have a greatly significant effect on the processing time.

The V4L2 encoder '''maxperf-enable''' is also available for the H265 encoder. The results for both resolutions are:

So here it seems that the performance is the same for both H265 and H264 codecs.

For these tests the main focus was to compare results between different configurations on the OMX and V4L2 encoders for H264 and H265 codecs. However, quality is hard to measure in an objective way, therefore we include some visual results for you to evaluate.

The tests were performed modifying the same configurations mentioned in the latency section and using the 1080p resolution.

We tried modifying the profiles on both encoders to see if the quality was significantly affected but the high profile which is the next one after the main profile did not seem to improve a lot the quality. Additionally, we noticed in the latency section that the high profile increases significantly the encoder processing time, so it is not a good alternative to improve quality.

<div><ul>  

</ul></div>

<div><ul>  

</ul></div>

The bitrate control was another relevant property to test, so we tried to set both encoders to use variable (VBR) and constant (CBR) bitrates, however it didn't seem to affect the H264 quality a lot.

<div><ul>  

</ul></div>

<div><ul>  

</ul></div>

On the other side, for H265 it seems that the V4L2 encoder quality is a little more affected by this property, while the OMX encoder does not seem to be affected at all.

<div><ul>  

</ul></div>

<div><ul>  

<li style="display: inline-block;"> [[File:V4l2-h265-vbr-cbr.gif|thumb|center|800px]] </li>

Another worthy test is the bitrate variation, since this parameter is directly associated with the video quality. Also, since we verified in the latency section that it did not seem to affect the processing time of the encoder, it would be a good alternative to improve quality without decreasing performance.

Here at the results we can notice that with lower bitrates the H264 encoder behaves better than H265.

<div><ul>  

</ul></div>

<div><ul>  

</ul></div>

The '''maxperf-enable''' property gives a really good boost to the V4L2 encoders in terms of latency, so it is important to validate if there's some tradeoff in quality. However, the results seem to tell that there is no difference in quality for both codecs just by enabling this property.

</ul></div>

Finally, we compare both OMX and V4L2 encoders. They both have similar results on H264, however in H265 it seems that V4L2 has a bit better quality.

</ul></div>

* The gst-omx is a plugin that wraps available [https://www.khronos.org/openmaxil OpenMAX IL] components and makes them available as standard GStreamer elements. This API allows library and codec implementers to rapidly and effectively utilize the full acceleration potential of new silicon, regardless of the underlying hardware architecture.

* As per the [https://developer.download.nvidia.com/embedded/L4T/r32_Release_v1.0/Docs/Tegra_Linux_Driver_Package_Release_Notes_R32.1.pdf?CnJhU5eVonboQmf3dQny80zS4QhQxnvbmSQOcXtmZvd_I_-CscHsN3YKYP_vsGxZVXkk-1HsxpWMY5_pDo2fyyIsacQ4maziyNWs3nkUJa8rP2mll1ovxK15BGJJEWz3kZsUAU_bswYa5BVU-wQLVeoNXiCcakJ9VgEZdXLmG0_XsWywF0p5pADZxjtJT5EQq8PJ5Q&t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczpcL1wvd3d3Lmdvb2dsZS5jb21cLyJ9 release notes] of '''Jetpack 4.2''', gst-omx plugins are deprecated and are said to be removed in future releases. Starting from this same release NVIDIA provides the gst-v4l2 plugin to use instead, which has proven to have a better performance.

* The minimum mean time achieved for OMX encoders on Jetpack 3.3.3 for a 1080p resolution is '''~22ms''', which means that 50 FPS are not viable.

|}

[[Category: Jsalas Notes]][[Category:Jetson]][[Category:GStreamer]]