GStreamer pipelines for Jetson TX2: Difference between revisions

{{GStreamer debug}}

On this page you are going to find a set of pipelines used on Jetson TX2, specifically used with the Jetson board. The wiki page tries to describe some of the multimedia features of the platform like the NVIDIA model to handle the ISP through its custom (and close) plugin called nvcamerasrc. It also includes a brief description of the sample application called nvgstcapture-1.0 and ways to measure the ARM load. The goal is to provide you some example pipelines and introduction to the board. It is a work in progress.

The camera sensor used for tests is the OV5693. These steps were run on Jetson TX2.

Recall that this board is a quad core A57 with some hardware units dedicated to encode/decode. Plugins accelerated by Hardware available:

</syntaxhighlight>

There is a tool to measure the ARM load created by NVIDIA for the Jetson TX2. This tool seems to give better statistics of the ARM cores than top. In the remaining sections of this wiki the ARM load reported by top is included for reference to the reader but tegrastats should contain the real load.

</syntaxhighlight>

This table tries to summarize the ARM load of different user cases of the Jetson TX2 using gstreamer and the capture plugin. The second column has the ARM load of the pipeline measured in one instant, using that value and the known value for the capture process the third column estimates the load of the encoding or display process (subtract the capture load). The load represent a global load of the system (4 ARMs) just to give an idea to the reader of the system capabilities.

|}

In order to test some of the videos or incoming streams produced by the pipelines below you might need VLC installed in your Jetson module, there is a known issue with VLC version 2.2 and you need to download and install VLC 2.4 or newer, in order to do it please follow these steps:

this issue has been reported on [https://devtalk.nvidia.com/default/topic/990198/?comment=5079474 NVIDIA's forum]

In order to capture there are two ways: Using v4l2 or using nvcamerasrc.

This plugin was created by nvidia and it has access to the ISP that helps converting from bayer to yuv suitable for the video encoders. However, it seems that its usage is limited to the ov5693 sensor until NVIDIA releases its source code or until it adds support to v4l2 to use the ISP.

</syntaxhighlight>

Starting on L4T R23.2 there is a /dev/video0 node to capture, however, this node will give you frames in bayer which are NOT suitable to encode because it grabs frames directly from the [https://www.ovt.com/sensors/OV5693 ov5693 camera] without using the ISP.

It works also with the UVC driver.

<syntaxhighlight lang='bash'>

</syntaxhighlight>

ARM load: 157% but only videotestsrc consumes 100%. According to the NVIDIA forum when using videotestsrc the system is also doing memory copies to feed the encoder with NVMM memory buffers so that loads more the ARM.

VLC can play the file correctly at 30fps and reports as codec: H264-MPEG-4 AVC (part 10) (h264). The fan on top of the SoC didn't turn on.

<syntaxhighlight lang='bash'>

</syntaxhighlight>

<syntaxhighlight lang='bash'>

</syntaxhighlight>

server:

vlc -vv --demux h264 --h264-fps 60  udp://@:5000  

</syntaxhighlight>

In case gstreamer complains when running the pipeline for missing the h264parse plugin, run the following command to install it:  

<syntaxhighlight lang='bash'>

</syntaxhighlight>

*1080p30

*Latency=138ms

ARM load: 156% but only videotestsrc consumes 100%. According to the NVIDIA forum when using videotestsrc the system is also doing memory copies to feed the encoder with NVMM memory buffers so that loads more the ARM.

VLC can play the file correctly at 30fps and reports as codec: Google/On2's VP8 Video (VP80). The fan on top of the SoC didn't turn on.

<syntaxhighlight lang='bash'>

</syntaxhighlight>

In order to support h265 you need VLC 2.2 in your computer to play the videos

</syntaxhighlight>

ARM load: 156% but only videotestsrc consumes 100%. According to the NVIDIA forum when using videotestsrc the system is also doing memory copies to feed the encoder with NVMM memory buffers so that loads more the ARM.

VLC can play the file correctly at 30fps and reports as codec: MPEG-H Part2/HEVC (H.265)(hevc). The fan on top of the SoC didn't turn on.

<syntaxhighlight lang='bash'>

</syntaxhighlight>

<syntaxhighlight lang='bash'>

</syntaxhighlight>

In order to start streaming H265 you need the h265 payloader, to do this you need to compile the latest version of gstreamer in your Jetson TX2 module. Please follow the [http://developer.ridgerun.com/wiki/index.php?title=Compiling_Tegra_X1/X2_source_code instructions to do it]. Same instructions can be run in your PC if you want to update the gstreamer version in your host. (in case of the host I had to recompile the good plugins to get the rtph265pay).

</syntaxhighlight>

*1080p30

*Latency=251ms

This is an application based on gstreamer and omx to capture, encode and save video to the filesystem. It has a lot of options. The source code is not released by NVIDIA yet but they are planning to release in future versions of Linux 4 Tegra (L4T), you can read about it at [https://devtalk.nvidia.com/default/topic/898129/jetson-tx1/enabling-camera-on-jetson-tx1-board/ Nvidia Embedded Computing Forum]

</syntaxhighlight>

Sometimes when the board has been just flashed you need to run reboot in order to run nvgstcapture-1.0 successfully. To take snapshots:

In general the application is consuming 15% of each of four cores. It is always displaying video this could affect.

<syntaxhighlight lang='bash'>

{{ContactUs}}