Jetson TX1/TX2 MultimediaAPI Memory Sharing

This page describes the memory sharing mechanisms between the different components in the Tegra X1/X2 Multimedia API.

Important Concepts

DMA Buffer

Linux kernel framework that provides a generic method for sharing buffers between multiple devices. A device can be either an Importer (consume buffers) or Exporter (produce buffers). In the scope of this wiki, when no direction is specified when referring to the DMA Buffer memory, Importer should be understood. Therefore, a DMA Buffer configured device will not allocate its own buffers, but consume external ones.

MMap

Unix-like framework to map devices and files to user space memory. In the context of this wiki, a MMap capable device is also a DMA Buffer Exporter. Therefore, a MMap configured device will allocate its own buffers.

User Pointer

V4L2 framework streaming mode where a device consumes memory from a pointer provided from user space. Therefore, a User Pointer configured device will not allocate its own buffers, but consume external ones.

EGLImage

EGL extension for contemporary mobile chipsets where the CPU and GPU share the same physical memory that allows direct access to the memory from both user space and GPU. In the context of this wiki, an EGLImage is create from an existing DMA Buffer.

Linux Memory

Memory accessible by a process at runtime. This memory can be either a HW buffer (like a DMA Buffer or MMapped Device) or custom allocated by the process via malloc, and similar.

Memory Sharing Matrix

The following table shows the different memory sharing mechanisms between the components available through the Multimedia API. This matrix does not consider format compatibility, see the next section.

		Direction	Output (Src)
		Component	Camera			Converter			Encoder	Decoder	CUDA	User Space
Direction	Component	Memory	DMA Buf	MMap Exporter	User Pointer	DMA Buf	MMap Exporter	User Pointer	MMap Exporter	MMap Exporter	EGLImage	Linux
Input (Sink)	Display	DMA Buf	NVBuf	YES	NVBuf	NVBuf	YES	NVBuf	YES	YES	NVBuf fd	NO
	Converter	DMA Buf	NVBuf	YES	NVBuf	NVBuf	YES	NVBuf	YES	YES	NVBuf fd	NO
		MMap Exporter	YES	NO	YES	YES	NO	YES	NO	NO	fd	NO
		User Pointer	NVBuf	YES	YES	YES	YES	NVBuf	YES	YES	NVBuf fd	YES
	Encoder	DMA Buf	NVBuf	YES	NVBuf	NVBuf	YES	NVBuf	YES	YES	NVBuf fd	NO
		MMap Exporter	YES	NO	NVBuf	YES	NO	YES	NO	NO	fd	NO
	Decoder	MMap Exporter	YES	NO	NVBuf	YES	NO	YES	NO	NO	fd	NO
	CUDA	EGLImage	NVBuf fd	fd	NVBuf fd	NVBuf fd	YES	NVBuf fd	fd	YES	YES	NO
	User Space	Linux	NVBuf	YES	YES	NVBuf	YES	YES	YES	YES	YES	YES

How to Read This Matrix

• The columns represent the component's output.
• The rows represent the component's input.
• fd is shorthand for NvEGLImageFromFd where a DMA buffer FD is wrapped into an EGLImage
• NVbuff is shorthand for NvCreateBuffer where a DMA buffer is allocated
• A component output is connected to the next component's input.
• The cells indicate if the memory mechanisms are compatible. For example:
  • The camera and the converter can share buffers via DMA Buffer, using NvCreateBuffer.
  • The converter can share MMap Buffers to the display DMA Buffer Importer.
  • The encoder cannot share buffers with the decoder, since both allocate via MMap

Format Support Matrix

IN PROGRESS