NVIDIA Jetson Orin/JetPack 5.0.2/Getting Started/Components: Difference between revisions

From RidgeRun Developer Wiki
(Created page with "<noinclude> {{NVIDIA Jetson Orin/Head|previous=JetPack 4.1‎/Getting Started/Wizard Flashing|next=JetPack 4.1‎/Compiling_Code|keywords=jetpack,component}} </noinclude> ==...")
 
mNo edit summary
 
(26 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<noinclude>
<noinclude>
{{NVIDIA Jetson Orin/Head|previous=JetPack 4.1‎/Getting Started/Wizard Flashing|next=JetPack 4.1‎/Compiling_Code|keywords=jetpack,component}}
{{NVIDIA Jetson Orin/Head|previous=JetPack 5.0/Getting Started/Wizard Flashing|next=JetPack 5.0/Compiling_Code|metakeywords=jetpack, component, Linux for Tegra, OpenCV, Vision Programming Interface, VPI, V4L2, CUDA Deep Neural Network, cuDNN, L4T, JetPack 5.0, Orin AGX, TensorRT}}
</noinclude>
</noinclude>


== Introduction ==
{{DISPLAYTITLE:NVIDIA Jetson Orin - JetPack Components|noerror}}


This section provides you with an overview of the Jetson Orin camera Subsystem at the software level. Also, it makes a comparison among Jetpack 5.0, which is the Jetpack that is used in the Orin platform, and the previous Jetpack, which is version 4.6.1.
This section provides an overview of some of the main software components provided in Jetpack 5.0 for the Orin AGX.


== Jetpack Components Comparison ==


{| class="wikitable" style="margin-right: 22em;"
==Jetpack Components==
Table 1 provides the versions of the key packages included in Jetpack 5.0, and the version previously available in Jetpack 4.6.1:
<br>
 
 
{| class="wikitable" style="margin: auto;"
|+ Table 1: Version of Jetpack components
|-
|-
! Package  
! Package  
Line 18: Line 23:
| 34.1
| 34.1
| 32.7.1
| 32.7.1
|-
| '''CUDA'''
| 11.4
| 10.2
|-
| '''cuDNN'''
| 8.3.2 
| 8.2.1
|-
|-
| '''TensorRT'''
| '''TensorRT'''
Line 23: Line 36:
| 8.2.1
| 8.2.1
|-  
|-  
| '''cuDNN'''
| 8.3.2 
| 8.2.1
|-
| '''CUDA'''
| 11.4.6
| 10.2
|-
| '''OpenCV'''
| '''OpenCV'''
| 4.5.4 (Developer Preview)
| 4.5.4 (Developer Preview)
Line 40: Line 45:
|-
|-
|}
|}
<br>


== New feature for kernel 5.10 ==
=== L4T ===


The Linux for Tegra (L4T) 34.1 provides the Linux Kernel 5.10, a bootloader based on UEFI interface, Ubuntu 20.04 based root file system,gcc-9.3-glibc-2.31 toolchain for 64-bit kernel and user space, etc.
=== Linux for Tegra===


Features:
Linux for Tegra (L4T) is the component that provides the OS components (kernel, file system, drivers, firmware) and the bootloader. Jetpack 5.0 comes with L4T version 34.1, featuring:
* Support for Jetson AGX Orin Developer Kit
* Support to emulate performance of production Jetson Orin NX series and Jetson AGX Orin series modules with Jetson AGX Orin Developer Kit
* LTS Kernel 5.10
* Ubuntu 20.04
* UEFI bootloader
* Support for NVSCI: Provides utilities for streaming data packets between different applications and for inter-process communication (IPC)
* New Jetson Power GUI: Tool for monitoring the power and thermal status of Jetson platform


=== TensorRT ===
* '''Linux Kernel version 5.10''': the previous Jetpack version (4.6) came with Linux Kernel version 4.9.
* '''Ubuntu 20.04-based root file system''': the previous Jetpack version (4.6) came with Ubuntu 18.04-based root file system.
* '''UEFI-based bootloader''': CBoot was used prior to L4T 34.1.
<!--* '''Jetson Orin NX performance emulation on Jetson AGX Orin''':-->
* '''Support for NVSCI''': Provides utilities for streaming data packets between different applications and for inter-process communication (IPC).
* '''Toolchain''': aarch64  gcc-9.3-glibc-2.31 for 64-bit Kernel and user space.
* '''Jetson Power GUI''': a new tool for monitoring the power and thermal status of the Jetson platform.
<br>
 
==== Multimedia API ====
Included as part of L4T, the Multimedia API allows developers to develop camera-based multimedia applications. It has two main components:
<br>
 
 
* V4L2 Sensor driver API: the V4L2 API that enables the capture devices, codecs, and scaling functionality.
* Libargus: the proprietary NVIDIA library that allows using the Jetson Image Signal Processor (ISP).
<br>


TensorRT is a framework created and published by NVIDIA for deep learning inference runtime. Jetpack 5.0 Developer Preview includes TensorRT 8.4.0 Early Access (EA). This EA is for early testing and feedback. For production, the documentation recommends continuing to use TensorRT 8.2.3 or later TensorRT 8.2.x patch.


TensorRT 8.4.0 EA has been tested with the following:
JetPack 5.0 Multimedia API features new NVUtils Multimedia buffer management and buffer transformation API. Previous nvbuf_utils is deprecated in this release.
<br>


* cuDNN 8.3.2
* TensorFlow 1.15.5
* PyTorch 1.9.0


=== CUDA ===
=== CUDA ===
CUDA Toolkit provides a comprehensive development environment for C and C++ developers building GPU-accelerated applications. The toolkit includes a compiler for NVIDIA GPUs, math libraries, and tools for debugging and optimizing the performance of your applications.
Jetpack 5.0 includes CUDA 11.4 support. CUDA is a parallel computing API for NVIDIA GPUs. NVIDIA's CUDA Toolkit provides the development environment for creating GPU-accelerated applications in C/C++. Starting with CUDA 11, the various components in the toolkit are versioned independently. Table 2 provides the CUDA 11.4 components version information:
<br>


Starting with CUDA 11, the various components in the toolkit are versioned independently.


For CUDA 11.4, the table below indicates the versions:
{| class="wikitable" style="margin: auto;"
{| class="wikitable" style="margin-right: 22em;"
|+ Table 2: CUDA 11.4 components version
|-
|-
! Component Name
! Component Name
Line 215: Line 225:
|}
|}


The last update of CUDA 11.4 is Update 4, in which the Linux x86_64 Driver Version is >=470.82.01.
<!--The last update of CUDA 11.4 is Update 4, in which the Linux x86_64 Driver Version is >=470.82.01.-->
<br>


=== Multimedia API ===
===CUDA Deep Neural Network (cuDNN)===
cuDNN is an NVIDIA library used by deep learning frameworks to accelerate common components, such as pooling, normalization, and backward convolution. Jetpack 5.0 comes with cuDNN 8.3.2, including bugfixes and new enhancements.
<br>


The Jetson Linux API Reference describes several APIs or API extensions that are implemented in NVIDIA® Jetson™ Linux Driver Package (L4T). You can use these APIs to customize aspects of your Jetson device's behavior.


Camera application API: libargus offers a low-level frame-synchronous API for camera applications, with per frame camera parameter control, multiple (including synchronized) camera support, and EGL stream outputs. RAW output CSI cameras needing ISP can be used with either libargus or GStreamer plugin. In either case, the V4L2 media-controller sensor driver API is used.
=== TensorRT ===


The Multimedia APIs are separate from the GStreamer framework, which provides high-level APIs. That framework is included in current and previous releases.
TensorRT is a framework created and published by NVIDIA for deep learning inference runtime. Jetpack 5.0 Developer Preview includes TensorRT 8.4.0 Early Access (EA). This EA is for early testing and feedback. For production, the documentation recommends continuing to use TensorRT 8.2.3 or later TensorRT 8.2.x patch. This version of TensorRT has support for CUDA 11.x.


JetPack 5.0 Developer Preview Multimedia highlights include:
TensorRT 8.4.0 EA has been tested with the following dependencies versions:  


* New NVUtils Multimedia buffer management and buffer transformation API.
* cuDNN 8.3.2
 
* TensorFlow 1.15.5
=== Camera Interface ===
* PyTorch 1.9.0
TBD (To Be Defined)
<br>
 
== Drivers comparasion ==
Comparison between imx219.c imx214.c drivers in Jetpack 5.0 and Jetpack 4.6.1, if the information is provided.
 
In the driver imx219.c added #if LINUX_VERSION_CODE and deleted #include <linux/version.h>
But in both drivers, the logic of the drivers is the same.
 
== Difference between Kernel 4.9  to Kernel 5.10 ==
 
=== Configuration Changes ===
The principal changes are resumed in the next table.
 
{| class="wikitable" style="margin-right: 22em;"
|-
! Environment variable
! Kernel 4.9
! Kernel 5.10
|-
| CONFIG_MEDIA
| N/A
| <ul><li> CONFIG_MEDIA_PLATFORM_SUPPORT=y
      <li> CONFIG_MEDIA_TEST_SUPPORT=y
  </ul>
|-
| CONFIG_TEGRA_GRHOST
| N/A
| <ul><li> CONFIG_TEGRA_GRHOST_NVCSI=y
      <li> CONFIG_TEGRA_GRHOST_SLVSEC=y
  </ul>
|-
| CONFIG_VIDEO
| N/A
| <ul><li> CONFIG_VIDEO_ISC=m
      <li> CONFIG_VIDEO_CDI=m
  </ul>
|-
| CONFIG_VIDEO (sensor)
| <ul><li> CONFIG_VIDEO_IMX185=m
      <li> CONFIG_VIDEO_IMX268=y
      <li> CONFIG_VIDEO_IMX318=y
      <li> CONFIG_VIDEO_OV9281=y
      <li> CONFIG_VIDEO_OV10823=y
      <li> CONFIG_VIDEO_OV23850=y
      <li> CONFIG_VIDEO_IMX390=y
      <li> CONFIG_VIDEO_IMX219=y
      <li> CONFIG_VIDEO_IMX274=m
      <li> CONFIG_VIDEO_LC898212=y
      <li> CONFIG_VIDEO_OV5693=mSen
  </ul>
| <ul><li> CONFIG_NV_VIDEO_IMX185=m
      <li> CONFIG_NV_VIDEO_IMX268=m
      <li> CONFIG_NV_VIDEO_IMX318=m
      <li> CONFIG_NV_VIDEO_OV9281=m
      <li> CONFIG_NV_VIDEO_OV10823=m
      <li> CONFIG_NV_VIDEO_OV23850=m
      <li> CONFIG_NV_VIDEO_IMX390=y
      <li> CONFIG_NV_VIDEO_IMX219=m
      <li> CONFIG_NV_VIDEO_IMX274=m
      <li> CONFIG_NV_VIDEO_LC898212=m
      <li> CONFIG_NV_VIDEO_OV5693=m
  </ul>
|-
| CONFIG_THERMAL_GOV
| CONFIG_THERMAL_GOV_CONTINUOUS=y
| N/A
|-
|}


=== Sensor Driver File Name Changes ===
=== OpenCV ===
The drivers file name in the Kernel 4.9 are imx185.c, imx268.c, imx318.c, ov9281.c, ov10823.c, ov23850.c, imx390.c, imx219.c, imx274.c, lc898212, ov5693.c, but in the Kernel 5.10 the prefix is ​​added nv_ before the drive file name.


For example, the imx219 driver name in kernel 4.9 is imx219.c, but the same driver in the kernel 5.10, the file name is nv_imx219.c
OpenCV is an Open Source library for computer vision tasks. Jetpack 5.0 supports OpenCV version 4.5.4.
<br>


=== Guarding Kernel Version-Specific Code ===


NVIDIA-specific kernel code in kernel/nvidia/ is guarded by checks that compare the LINUX_VERSION_CODE macro against KERNEL_VERSION(x, x, x).
=== Vision Programming Interface (VPI) ===
 
VPI is a library provided by NVIDIA to easily take advantage of the power of the Programmable Vision Accelerator and the GPU. It brings Computer Vision and Image Processing algorithms implemented to run the PVA, OFA (Optical Flow Accelerator), GPU, and CPU. Jetpack 5.0 comes with VPI 2.0 version. To learn more about how you can use VPI along with GStreamer, visit our [[NVIDIA_Jetson_Orin/RidgeRun_Products/GstVPI|GstVPI]] section.
Also, there are important changes in the new driver by comparing the following macros:
<br>
 
The use of powergate_id is dropped, as tegra_powergate is deprecated in kernel version 5.10. The following table shows the paths of the kernel 5.10 and the previous kernel.
 
{| class="wikitable" style="margin-right: 22em;"
|-
! Kernel 4.9
! Kernel 5.10
|-
| kernel/kernel-4.9/include/linux/tegra_pm_domains.h
| kernel/kernel-5.10/include/linux/pm_domain.h
|-
|}
 
To replace the video buffer state, the V4L2 framework changes by adapting the VI drivers. Also, by doing this adaptation, V4L2 framework changes to time related v4l2-ctl APIs and structures. Some examples provided by NVIDIA are shown in the following table:
 
{| class="wikitable" style="margin-right: 22em;"
|-
! Kernel 4.9
! Kernel 5.10
|-
| getrawmonotonic()
| ktime_get_ts64 
|-
| struct timespec
| struct timespec64
|-
| ns_to_timespec
| ns_to_timespec64
|-
| VB2_BUF_STATE_REQUEUEING
| VB2_BUF_STATE_ERROR
|-
| VFL_TYPE_VIDEO
kernel/kernel-4.9/​include/media/v4l2-dev.h
| VFL_TYPE_GRABBER
kernel/kernel-5.10/​include/media/v4l2-dev.h 
|-
| v4l2_mbus_type for kernel version 4.9:
kernel/kernel-4.9/
​include/media/v4l2-mediabus.h
| Modified enum type for C-PHY and D-PHY in v4l2_mbus_config:
/dev-main_l4t/kernel/kernel-5.10/
​include/media/v4l2-mediabus.h
|}
 
 
=== I2C API and dev_err() Function===
I2C API and dev_err() function changes as can be seen in the following comparative table.
{| class="wikitable" style="margin-right: 22em;"
|-
! Kernel 4.9
! Kernel 5.10
|-
| kernel/kernel-4.9/​include/linux/device.h
| kernel/kernel-5.10/​include/linux/dev_printk.h
|-
| <ul> <li> i2c_lock_adapter() and i2c_unlock_adapter()
    <li> kernel/kernel-4.9/​include/linux/i2c.h
  </ul>
| <ul><li> i2c_lock_bus() and i2c_unlock_bus()
      <li> kernel/kernel-5.10/​include/linux/i2c.h
  </ul>
|-
| <ul><li> i2c_new_device()
    <li> kernel/kernel-4.9/​drivers/i2c/i2c-core.c
  </ul>
| <ul><li> i2c_new_client_device()
    <li> kernel/kernel-5.10/​drivers/i2c/i2c-core-base.c
  </ul>
|-
|}


=== NVIDIA Capture Driver Code Path ===


The new capture mode driver file paths for channel configuration (VI, ISP) is
<pre>
kernel/nvidia/drivers/​media/platform/tegra/camera/fusa-capture/
</pre>


Also, the driver for the VI is the vi5.


<noinclude>
<noinclude>
{{NVIDIA Jetson Orin/Foot|JetPack 4.1‎/Getting Started/Wizard Flashing|JetPack 4.1‎/Compiling_Code}}
{{NVIDIA Jetson Orin/Foot|JetPack 5.0‎/Getting Started/Wizard Flashing|JetPack 5.0‎/Compiling_Code}}
</noinclude>
</noinclude>

Latest revision as of 17:45, 10 February 2023


Previous: JetPack 5.0/Getting Started/Wizard Flashing Index Next: JetPack 5.0/Compiling_Code






This section provides an overview of some of the main software components provided in Jetpack 5.0 for the Orin AGX.


Jetpack Components

Table 1 provides the versions of the key packages included in Jetpack 5.0, and the version previously available in Jetpack 4.6.1:


Table 1: Version of Jetpack components
Package Jetpack 5.0 Version Jetpack 4.6.1 Version
Linux for Tegra 34.1 32.7.1
CUDA 11.4 10.2
cuDNN 8.3.2 8.2.1
TensorRT 8.4.0 Early Access 8.2.1
OpenCV 4.5.4 (Developer Preview) 4.1.1
Vision Programing Interface (VPI) 2.0 1.2



Linux for Tegra

Linux for Tegra (L4T) is the component that provides the OS components (kernel, file system, drivers, firmware) and the bootloader. Jetpack 5.0 comes with L4T version 34.1, featuring:

  • Linux Kernel version 5.10: the previous Jetpack version (4.6) came with Linux Kernel version 4.9.
  • Ubuntu 20.04-based root file system: the previous Jetpack version (4.6) came with Ubuntu 18.04-based root file system.
  • UEFI-based bootloader: CBoot was used prior to L4T 34.1.
  • Support for NVSCI: Provides utilities for streaming data packets between different applications and for inter-process communication (IPC).
  • Toolchain: aarch64 gcc-9.3-glibc-2.31 for 64-bit Kernel and user space.
  • Jetson Power GUI: a new tool for monitoring the power and thermal status of the Jetson platform.


Multimedia API

Included as part of L4T, the Multimedia API allows developers to develop camera-based multimedia applications. It has two main components:


  • V4L2 Sensor driver API: the V4L2 API that enables the capture devices, codecs, and scaling functionality.
  • Libargus: the proprietary NVIDIA library that allows using the Jetson Image Signal Processor (ISP).



JetPack 5.0 Multimedia API features new NVUtils Multimedia buffer management and buffer transformation API. Previous nvbuf_utils is deprecated in this release.


CUDA

Jetpack 5.0 includes CUDA 11.4 support. CUDA is a parallel computing API for NVIDIA GPUs. NVIDIA's CUDA Toolkit provides the development environment for creating GPU-accelerated applications in C/C++. Starting with CUDA 11, the various components in the toolkit are versioned independently. Table 2 provides the CUDA 11.4 components version information:


Table 2: CUDA 11.4 components version
Component Name Version Information Supported Architectures
CUDA Runtime (cudart) 11.4.148 x86_64, POWER, Arm64
cuobjdump 11.4.120 x86_64, POWER, Arm64
CUPTI 11.4.120 x86_64, POWER, Arm64
CUDA cuxxfilt (demangler) 11.4.120 x86_64, POWER, Arm64
CUDA Demo Suite 11.4.100 x86_64
CUDA GDB 11.4.120 x86_64, POWER, Arm64
CUDA Memcheck 11.4.120 x86_64, POWER
CUDA NVCC 11.4.152 x86_64, POWER, Arm64
CUDA nvdisasm 11.4.152 x86_64, POWER, Arm64
CUDA NVML Headers 11.4.120 x86_64, POWER, Arm64
CUDA nvprof 11.4.120 x86_64, POWER, Arm64
CUDA nvprune 11.4.120 x86_64, POWER, Arm64
CUDA NVRTC 11.4.152 x86_64, POWER, Arm64
CUDA NVTX 11.4.120 x86_64, POWER, Arm64
CUDA NVVP 11.4.193 x86_64, POWER
CUDA Samples 11.4.120 x86_64, POWER, Arm64
CUDA Compute Sanitizer API 11.4.120 x86_64, POWER, Arm64
CUDA Thrust 11.4.122 x86_64, POWER, Arm64
CUDA cuBLAS 11.6.5.2 x86_64, POWER, Arm64
CUDA cuFFT 10.5.2.100 x86_64, POWER, Arm64
CUDA cuFile 1.0.2.10 x86_64
CUDA cuRAND 10.2.5.120 x86_64, POWER, Arm64
CUDA cuSOLVER 11.2.0.120 x86_64, POWER, Arm64
CUDA cuSPARSE 11.6.0.120 x86_64, POWER, Arm64
CUDA NPP 11.4.0.110 x86_64, POWER, Arm64
CUDA nvJPEG 11.5.2.120 x86_64, POWER, Arm64
Nsight Compute 2021.2.2.1 x86_64, POWER, Arm64 (CLI only)
NVTX 1.21018621 x86_64, POWER, Arm64
Nsight Systems 2021.3.2.4 x86_64, POWER, Arm64 (CLI only)
Nsight Visual Studio Edition (VSE) 2021.2.1.21205 x86_64 (Windows)
nvidia_fs1 2.7.53 x86_64
Visual Studio Integration 11.4.120 x86_64 (Windows)
NVIDIA Linux Driver 470.82.01 x86_64, POWER, Arm64
NVIDIA Windows Driver 472.50 x86_64 (Windows)


CUDA Deep Neural Network (cuDNN)

cuDNN is an NVIDIA library used by deep learning frameworks to accelerate common components, such as pooling, normalization, and backward convolution. Jetpack 5.0 comes with cuDNN 8.3.2, including bugfixes and new enhancements.


TensorRT

TensorRT is a framework created and published by NVIDIA for deep learning inference runtime. Jetpack 5.0 Developer Preview includes TensorRT 8.4.0 Early Access (EA). This EA is for early testing and feedback. For production, the documentation recommends continuing to use TensorRT 8.2.3 or later TensorRT 8.2.x patch. This version of TensorRT has support for CUDA 11.x.

TensorRT 8.4.0 EA has been tested with the following dependencies versions:

  • cuDNN 8.3.2
  • TensorFlow 1.15.5
  • PyTorch 1.9.0


OpenCV

OpenCV is an Open Source library for computer vision tasks. Jetpack 5.0 supports OpenCV version 4.5.4.


Vision Programming Interface (VPI)

VPI is a library provided by NVIDIA to easily take advantage of the power of the Programmable Vision Accelerator and the GPU. It brings Computer Vision and Image Processing algorithms implemented to run the PVA, OFA (Optical Flow Accelerator), GPU, and CPU. Jetpack 5.0 comes with VPI 2.0 version. To learn more about how you can use VPI along with GStreamer, visit our GstVPI section.




Previous: JetPack 5.0‎/Getting Started/Wizard Flashing Index Next: JetPack 5.0‎/Compiling_Code