User Guide-Integration Options
🚧 Documentation under development
The PVA ISP for NVIDIA Jetson guide is currently under active development. Some sections may be incomplete or change without notice.
Questions? Contact RidgeRun or email to support@ridgerun.com.
PVA-ISP integration options
PVA-ISP can be integrated in a product in two main ways: through a GStreamer element or directly through a C++ application using the PVA-ISP library. Both options use the same underlying ISP algorithms and PVA execution model. The complete ISP flow is:
The processing stages are backed by PVA kernels and are orchestrated by the
PVA-ISP pipeline runtime. The full application flow can also expose a histogram
branch after Black Level Correction and a scene-key branch from Demosaic into
Global Tone Mapping. Moreover, some stages can be removed in order to reduce latency if
the application supports it.
GStreamer element
The GStreamer integration exposes PVA ISP as a standard GstCUDA-based GStreamer element for multimedia applications built on GStreamer pipelines. It is suitable for use cases such as live video streaming, video recording, display pipelines, and AI inference workflows using NVIDIA DeepStream. The element receives Bayer image data, executes the configured ISP pipeline, and produces processed output for downstream GStreamer elements.
The current implementation accepts Bayer input and produces NV12 output.
The element supports standard GStreamer system-memory buffers on its sink and source pads. In particular, the sink pad accepts video/x-bayer, while the source pad can expose video/x-raw NV12. It also provides zero-copy as in GstCUDA.
GStreamer Integration Considerations
The GStreamer element supports Bayer input and NV12 output using system memory, provided by a CUDA Host memory pool, whose memory is accessible via CPU or PVA with zero copy.
For Bayer capture pipelines, this is important because many sensors produce RAW formats above 8 bits per pixel, such as 10-bit, 12-bit, 14-bit, or 16-bit Bayer. In those cases, platform-specific handling may be required to preserve the Bayer data correctly across the multimedia pipeline.
The GStreamer element is the recommended integration method when the application already uses GStreamer for capture, processing, encoding, streaming, display, or AI inference.
Applications that need direct control over graph construction, frame ownership, or ISP stage configuration may prefer the C++ library API. For additional information about building custom ISP pipelines using the library API, see Custom_Development.
C++ application or library integration
The C++ option links the product application directly against the PVA-ISP
library, currently built as libpva_isp.so. This is the natural
integration path when the application already controls capture, buffer
allocation, parameter loading, frame scheduling, or product-specific logic in
C++.
In this model, the application builds the ISP graph using the public API. The pipeline runtime owns the stage graph, negotiated frames, internal buffers, and shared PVA execution context. The application decides which stages are present, which pads are connected, which boundary frames are externally visible, and which properties are applied.
The typical lifecycle is:
isp::Pipeline pipeline{};
auto* dec = pipeline.addStage(isp::StageType::kDecompand, "Decompand");
auto* blc = pipeline.addStage(isp::StageType::kBlackLevel,
"Black Level Correction");
auto* awb = pipeline.addStage(isp::StageType::kWhiteBalance,
"White Balance");
auto* dmc = pipeline.addStage(isp::StageType::kDemosaic, "Demosaic");
auto* gtm = pipeline.addStage(isp::StageType::kGlobalToneMapping,
"Global Tone Mapping");
auto* gamma = pipeline.addStage(isp::StageType::kGammaToneMapping,
"Gamma Tone Mapping");
auto* nv12 = pipeline.addStage(isp::StageType::kNV12Conversion,
"NV12 Conversion");
pipeline.setInput("Decompand", bayer_input_spec);
pipeline.setOutput("NV12 Conversion", nv12_output_spec);
pipeline.linkMany({dec, blc, awb, dmc, gtm, gamma, nv12});
pipeline.link(dmc, "scene_key", gtm, "scene_key");
pipeline.prepare(/*exec_stats=*/true);
uint8_t* output = pipeline.process();
The library pipeline performs:
- Stage creation and ownership.
- Pad-based graph linking.
- Frame-spec negotiation between stages.
- Internal frame allocation.
- PVA program configuration.
- Ordered stage execution.
- Optional per-stage profiling statistics.
Pipeline stages
The implemented ISP stages are:
| Stage | Input | Output | Notes |
|---|---|---|---|
| Decompand | Bayer RAW | Linearized Bayer RAW | Expands companded sensor values using piecewise control points and a generated
VLUT. |
| Black Level Correction | Bayer RAW | Corrected Bayer RAW; optional gains side output
|
Subtracts per-channel optical black levels and applies digital gain. |
| Histogram | Corrected Bayer RAW | Histogram buffer | Computes histogram information from Bayer-domain luminance samples. |
| White Balance | Bayer RAW plus optional gains input
|
White-balanced Bayer RAW | Applies manual gains or gains computed by the previous stage. |
| Demosaic | Bayer RAW | Planar RGB plus scene_key side output
|
Converts Bayer to RGB and emits scene-key statistics for global tone mapping. |
| Global Tone Mapping | RGB plus scene_key
|
Tone-mapped RGB | Applies global tone mapping using brightness and temporal adaptation controls. |
| Gamma Tone Mapping | Tone-mapped RGB | Gamma-corrected RGB | Applies a gamma LUT generated during configuration. |
| NV12 Conversion | RGB | NV12 | Converts the final RGB output into NV12 for display, encoding, or streaming. |
More information on the stages is available in Supported Algorithms.
