What is PVA ISP?
What is RidgeRun PVA ISP?
PVA ISP is RidgeRun's hardware-accelerated Image Signal Processor (ISP) accelerated by NVIDIA's Programmable Vision Accelerator (PVA), available in NVIDIA Orin and Thor, providing a configurable PVA-accelerated image-processing pipeline that transforms raw sensor data into formats suitable for display, video encoding, streaming, and computer vision applications. All this without loading the GPU with ISP tasks.
PVA ISP is an alternative to NVIDIA's Argus hardware-accelerated ISP and is adequate for other capture systems, such as the Holoscan Sensor Bridge, where Argus is no longer used. Moreover, PVA ISP enables full pipeline visibility, configurable processing stages, access to intermediate results, and the flexibility to integrate custom algorithms and application-specific workflows.
Unlike traditional ISP implementations that operate as fixed-function pipelines, PVA ISP exposes image processing stages as configurable accelerated components, enabling customization, tuning, and future extensibility while leveraging dedicated vision acceleration hardware.
As computer vision systems become more specialized, developers increasingly require:
- Custom image processing algorithms
- Application-specific image tuning
- Access to intermediate processing stages
- Easier debugging and validation
- Extensible processing pipelines
PVA ISP was created to provide a flexible image processing framework capable of adapting to application-specific requirements while leveraging NVIDIA's dedicated vision processing hardware.
The current PVA ISP implementation supports the image processing stages shown below.
| Stage | Purpose |
|---|---|
| Decompanding | Expands compressed Bayer sensor data into a higher dynamic range using configurable lookup tables. Supports input bit depths from 10 to 16 bits. |
| Black Level Correction | Removes sensor black-level offsets and establishes a proper black reference level. |
| Digital Gain | Applies four configurable gains, one for each site in the Bayer 2×2 cell, before color reconstruction. |
| White Balance / Auto White Balance (AWB) | Adjusts color channel gains to correct color casts caused by varying illumination conditions. Supports manual gains and automatic gain estimation. |
| Demosaicing | Reconstructs full RGB color information from Bayer sensor measurements using a 5x5 Malvar-He-Cutler interpolation algorithm. |
| 3x3 Convolution | Applies configurable spatial filters for image sharpening, smoothing, anti-aliasing, and other filtering operations. |
| Color Correction Matrix (CCM) | Applies a configurable 3x3 color transformation matrix to improve color accuracy and reproduction. |
| Histogram Generation | Generates image statistics used for exposure control, image analysis, and tuning. |
| Auto Exposure (AE) | Uses image histogram statistics to calculate exposure adjustments and adapt to changing lighting conditions. |
| Global Tone Mapping (GTM) | Compresses image dynamic range while preserving visual detail, using scene statistics and configurable tone-mapping parameters. |
| DLUT-based Global Tone Mapping | Provides an alternative Global Tone Mapping implementation using the PVA's Decoupled Lookup Table (DLUT) capabilities for efficient tone mapping. |
| Gamma Tone Mapping | Applies configurable gamma correction using generated or custom lookup tables to adjust image brightness and tonal response. |
| RGB to NV12 Conversion | Converts RGB images into the NV12 format commonly used by video encoders and multimedia pipelines. |
In addition to standalone processing stages, PVA ISP supports joined-kernel stages that combine multiple ISP operations into a single PVA kernel. These stages reduce intermediate memory transfers and DMA overhead, improving processing efficiency.
The supported joined-kernel combinations include:
- Input Stage: Decompanding + Black Level Correction
- Pre RGB: White Balance + Demosaicing
- Convolution + Color Correction: 3x3 Convolution + Color Correction Matrix
- Post RGB: Global Tone Mapping + Gamma Tone Mapping + NV12 Conversion
- Post RGB DLUT: DLUT-based Global Tone Mapping + Gamma Tone Mapping + NV12 Conversion
Standalone and joined-kernel stages can be selected when constructing the ISP pipeline, allowing developers to balance processing flexibility and performance according to their application requirements.
For additional information about each stage, consult the Supported PVA ISP algorithms page.
Common Use Cases
PVA ISP is particularly well suited for applications where image processing behavior must be understood, validated, customized, or optimized for specific operating conditions.
These applications often require greater control than what is typically available through black-box ISP implementations and may involve challenging imaging environments, application-specific algorithms, or strict validation requirements.
-
Autonomous Robots -
Industrial Inspection -
Automotive Perception -
Smart Cities -
Security and Surveillance -
Drones and UAVs -
Medical Imaging Devices -
Agricultural Vision Systems
FAQ
- What problem does PVA ISP solve on Jetson?
- PVA ISP gives developers a programmable ISP pipeline that can offload image processing stages from the GPU to the PVA, which is useful when a product needs more control, visibility, or customization than the default camera stack provides.
- When should I use PVA ISP instead of Argus?
- Use PVA ISP when you need explicit control of ISP stages, intermediate results, custom tuning, or integration paths that do not fit the standard Argus camera workflow.
- Can I integrate PVA ISP with GStreamer?
- Yes. RidgeRun documents a GStreamer integration path for fast evaluation and pipeline-based product prototyping, and a C++ integration path for tighter application control.
- What should I send RidgeRun when requesting support?
- Include the Jetson module, JetPack/L4T version, camera sensor, Bayer format, target resolution and frame rate, desired output format, and the latency or throughput target for your use case.
- Is PVA ISP a replacement for every Jetson camera workflow?
- No. PVA ISP is most relevant when a team needs more control and observability than the default path provides.
