Jetson TX1/TX2 MultimediaAPI Memory Sharing

From RidgeRun Developer Wiki



This page describes the memory sharing mechanism between the different components in the Jetson TX1/TX2 Multimedia API.

Important Concepts

DMA Buffer
Linux kernel framework that provides a generic method for sharing buffers between multiple devices. A device can be either an Importer (consume buffers) or Exporter (produce buffers). In the scope of this wiki, when no direction is specified when referring to the DMA Buffer memory, Importer should be understood. Therefore, a DMA Buffer configured device will not allocate its own buffers but consume external ones.
MMap
Unix-like framework to map devices and files to user space memory. In the context of this wiki, a MMap capable device is also a DMA Buffer Exporter. Therefore, a MMap configured device will allocate its own buffers.
User Pointer
V4L2 framework streaming mode where a device consumes memory from a pointer provided from userspace. Therefore, a User Pointer configured device will not allocate its own buffers, but consume external ones.
EGLImage
EGL extension for contemporary mobile chipsets where the CPU and GPU share the same physical memory that allows direct access to the memory from both user space and GPU. In the context of this wiki, an EGLImage is created from an existing DMA Buffer.
Linux Memory
Memory accessible by a process at runtime. This memory can be either a HW buffer (like a DMA Buffer or MMapped Device) or custom allocated by the process via malloc, and similar.

Memory Sharing Matrix

The following table shows the different memory sharing mechanisms between the components available through the Multimedia API. This matrix does not consider format compatibility, see the next section.

Direction Output
(Src)
Component Camera Converter Encoder Decoder CUDA User Space
Direction Component Memory DMA Buf MMap
Exporter		 User
Pointer		 DMA Buf MMap
Exporter		 User
Pointer		 MMap
Exporter		 MMap
Exporter		 EGLImage Linux
Input
(Sink)		 Display DMA Buf NVBuf YES NVBuf NVBuf YES NVBuf YES YES NVBuf
fd		 NO
Converter DMA Buf NVBuf YES NVBuf NVBuf YES NVBuf YES YES NVBuf
fd		 NO
MMap
Exporter		 YES NO YES YES NO YES NO NO fd NO
User
Pointer		 NVBuf YES YES YES YES NVBuf YES YES NVBuf
fd		 YES
Encoder DMA Buf NVBuf YES NVBuf NVBuf YES NVBuf YES YES NVBuf
fd		 NO
MMap
Exporter		 YES NO NVBuf YES NO YES NO NO fd NO
Decoder MMap
Exporter		 YES NO NVBuf YES NO YES NO NO fd NO
CUDA EGLImage NVBuf
fd		 fd NVBuf
fd		 NVBuf
fd		 YES NVBuf
fd		 fd YES YES NO
User Space Linux NVBuf YES YES NVBuf YES YES YES YES YES YES

How to Read This Matrix

  • The columns represent the component's output.
  • The rows represent the component's input.
  • fd is shorthand for NvEGLImageFromFd where a DMA buffer FD is wrapped into an EGLImage
  • NVbuff is shorthand for NvCreateBuffer where a DMA buffer is allocated
  • A component output is connected to the next component's input.
  • The cells indicate if the memory mechanisms are compatible. For example:
    • The camera and the converter can share buffers via DMA Buffer, using NvCreateBuffer.
    • The converter can share MMap Buffers to the display DMA Buffer Importer.
    • The encoder cannot share buffers with the decoder, since both allocate via MMap

Format Support Matrix

IN PROGRESS

For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.


Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.

COMPANY

SUPPORT

ARTIFICIAL INTELLIGENCE

© 2024 RidgeRun. All rights reserved.


Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Jetson_TX1/TX2_MultimediaAPI_Memory_Sharing&oldid=42765"