NVIDIA Jetson AGX Thor - Video Capture and Display - Cameras - MIPI CSI
 | The NVIDIA Jetson AGX Thor documentation from RidgeRun is presently being developed. |
Overview
The Camera Serial Interface (CSI) is a standardized protocol defined by the Mobile Industry Processor Interface (MIPI) Alliance for connecting camera sensors to SoCs using high-speed serial links. CSI is widely used across embedded, robotics, and mobile systems.
| Advantages | Limitations |
|---|---|
| • High bandwidth per pin • Low power usage • Differential signaling (low EMI) • Compact wiring • Broad format support • Virtual channel multiplexing • Linux & BSP ecosystem support |
• Cable length limited (≤ 20 cm) • Not hot-pluggable • Signal integrity critical • No inline debugging • No flow control/retry • SoC/sensor configuration tightly coupled |
Protocol versions
- CSI-1: Legacy, rarely used today.
- CSI-2: Most widely adopted. Supported by Jetson AGX Thor.
- CSI-3: Bidirectional UniPro-based M-PHY, but rarely implemented.
Physical layers
CSI-2 supports two different physical layers D-PHY and C-PHY. Both standard physical layers are also set by the MIPI Alliance.
- D-PHY: Uses differential signaling, with a clock signal and N datalanes. It is the most common implementation.
- C-PHY: Uses 3-phase signaling. Allows for higher performance per wire.
CSI integration workflow
Jetson platforms with MIPI CSI support follow a layered architecture that spans from hardware to user applications. A typical CSI pipeline captures image data from a sensor, transmits it through the SoC's CSI receiver, optionally processes it through an ISP or GPU, and exposes it to software through standard Linux interfaces like V4L2 or Jetson's Camera Core Library drivers.
The Jetson MIPI CSI pipeline consists of the following integration layers, where each depends on the correct setup of the previous:
1. Hardware Integration: Includes the camera module (e.g., MIPI sensor or HDMI-to-CSI bridge), the CSI receiver block in the Jetson SoC, power rails and control lines (I²C, GPIOs).
2. Hardware Description: Implemented via the Device Tree, which maps sensor connections, lane configuration, addresses, and CSI topology.
3. Driver Binding: The kernel probes and binds sensor and CSI drivers, and registers corresponding media entities.
4. Media Graph Construction: The Linux media controller links all components into a unified capture pipeline: Sensor → CSI → ISP (optional) → DMA output.
5. User-Space Interface: Applications access the pipeline through /dev/videoX using V4L2 APIs or high-level libraries like Jetson Camera Core.
Hardware Setup
A standard MIPI CSI interface consists of the following signals and power lines:
- 1 differential clock lane for synchronization
- 1-4 (or more) differential data lanes for video transmission
- I²C control bus for sensor register access and initialization
- Multiple power rails (analog, digital, and I/O)
- Optional GPIOs for reset, power-down, and interrupts
CSI lanes must be impedance-matched differential pairs with low skew. Polarity and lane order must align with the receiver or the link may fail.
Device Tree Description
In the context of CSI integration, the Device Tree (DT) acts as the hardware description layer that maps camera sensors to the CSI receiver. It informs the kernel about how devices are connected and which drivers to bind, allowing the media pipeline to be constructed correctly at boot.
Each sensor must define basic properties such as its I²C address, CSI port and lane configuration, and connection to the CSI receiver via endpoint nodes. This configuration enables the kernel to register the sensor and CSI blocks as media entities, forming the foundation of the capture pipeline.
Driver binding and initialization
At boot time, the Linux kernel probes the sensor and CSI interfaces based on the DT definition. The initialization process involves:
- The sensor driver communicating over I²C to detect and configure the camera.
- The CSI receiver driver setting up the physical D-PHY and internal routing.
- The media controller framework registering entities and building the media graph linking the sensor, CSI block, and downstream interfaces
If either the sensor or CSI receiver driver fails to load, the video pipeline will not be available.
Capture Node Setup and Streaming
Once the media graph is successfully configured, the camera is exposed to user space as a video capture node, typically /dev/video0. Streaming is performed using memory-mapped buffers or DMA, in a selected resolution and pixel format supported by both the sensor and CSI receiver. Pixel data may be streamed in raw formats (e.g., Bayer RAW10) or processed through the ISP to output RGB or YUV. Tools like v4l2-ctl, GStreamer, or frame grabbers can be used to test, visualize, and validate the video stream.
Camera driver development process
Core Components of a CSI driver solution
A MIPI CSI driver stack is composed of several elements:
- Driver Interface Layer - Defines whether the system uses a high-level camera core library or direct V4L2 bindings.
- Sensor Subdevice Driver - Implements control of the camera over I²C/SPI and formats like exposure, resolution, and frame rate.
- CSI Receiver Support - Ensures synchronization with the CSI-2 bus, including D-PHY timing, lane mapping, and error checking.
- Media Controller Integration - Constructs and manages the media graph: sensor → CSI → (optional ISP) → output node.
- Device Tree Description - Expresses hardware wiring, GPIO controls, and the CSI link configuration.
- Acceleration and Processing - Optionally routes output through ISP, GPU, or hardware encoders for post-processing.
- Validation and Tuning Tools - Includes test pipelines, logging, error metrics, and regression workflows.
Architecture Models
When developing for a Jetson device there are two primary high-level software interface models available for camera handling:
Camera Core Library
Ideal when image post-processing for Bayer sensors, ISP control, or format conversion are required (RGB to YUV). This layer abstracts application development while operating over underlying kernel V4L2 drivers.
Direct V4L2 Interface
Provides unfiltered access to the CSI camera pipeline via standard V4L2 IOCTL calls. Useful for validating sensor behavior, handling raw Bayer data, or debugging.
CSI Camera stack diagram
The following diagram is a simplified representation of the software stack on a CSI camera implementation on Jetson:
Kernel Driver Development
Jetson camera sensors using MIPI CSI are typically implemented as V4L2 subdevices following the V4L2 and media controller frameworks. These drivers expose a low-level interface for sensor control and video configuration, while delegating pixel transport to the CSI receiver.
Core Responsibilities
A sensor subdevice driver must implement the following:
- Power management — including initialization and shutdown sequences via runtime PM hooks for enabling regulators, clocks, and GPIOs.
- Streaming controls — enabling and disabling data flow and handlers to control frame flow.
- Format negotiation — to support resolutions, frame rates, and pixel formats.
- Pad operations — sensor drivers typically expose one source pad with media bus format negotiation.
- Mode tables — a static array of supported resolutions, pixel formats, and frame rates, mapped to register configurations.
- Control handling — for exposure, gain, test pattern, and other runtime controls.
The driver must also register the subdevice and initialize media entity pads for graph linking.
Device tree development
A complete Device Tree configuration is vital for the system to recognize and initialize the camera hardware. The Device Tree serves as the hardware abstraction layer, describing the physical connections, control interfaces, and data paths between the sensor, the CSI receiver, and the rest of the system.
A top-level camera module node groups one or more imaging devices, includes physical characteristics (e.g., identifier, orientation, position), and links driver bindings.
Each individual sensor node must define the necessary parameters for hardware-level integration. These include:
- I²C bus and address for sensor communication
- CSI port ID, number of data lanes, and clock lane mapping
- Supported formats, resolutions, and pixel clock settings
- Link frequency and signal polarities
- Virtual channel ID (for multi-stream configurations)
- Optional PHY timing overrides
- Node activation via a
status = "okay"flag. - CSI endpoint details: clock lanes, data lanes, data type, and virtual channel mapping.
- Power and reset GPIOs.
During boot, the platform camera framework parses this structure and registers the camera as a media entity. This process allows the sensor driver, CSI receiver, and any processing units (like the ISP) to form a complete and coherent media graph, enabling discovery, configuration, and streaming.
Validation, tuning, and production readiness
After driver integration, validation processes and tests must be performed to ensure the correct behavior and product quality of the driver, some examples are:
- Initial validation involves loading the driver and confirming buffer negotiation, frame continuity, and format accuracy via tools such as v4l2-ctl or graphical pipelines.
- Performance measurement includes assessing bandwidth, frame rates, and ensuring that the system functions without buffer overruns or synchronization errors.
- Sensor tuning, when using Bayer-type cameras, may require calibration of color balance, exposure, noise handling, and optional ISP parameter tuning with specialized tools.
- Endurance testing to ensure correct capture over long periods of time.
What RidgeRun can offer
VALIDATE
RidgeRun specializes in advanced multimedia and embedded Linux development, with deep experience in Jetson platforms and camera integration across a range of applications including robotics, AI, industrial vision, automotive systems and more.
- Sensor Driver Development: Custom V4L2 subdevice drivers for MIPI CSI and bridge-based cameras (HDMI-to-CSI, LVDS-to-CSI), including support for both Bayer and YUV sensors.
- GStreamer Pipeline Integration: RidgeRun develops and maintains GStreamer plugins (e.g., GstVPI, GstCUDA) and can integrate your camera into optimized zero-copy pipelines with encoder, inference, or display paths.
- Jetson ISP Tuning: Support for Bayer sensor calibration and ISP configuration using NVIDIA's tuning toolchain.
- Performance Optimization: CSI and memory bandwidth profiling, DMA throughput testing, and integration with tools like Nsight Systems.
- Production-Grade Validation: Automated testing, regression frameworks, and long-term BSP maintenance.
We also offer BSP customization, bootloader integration, DeepStream deployment, and advanced image processing using VPI and CUDA.
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
