Examples - Sample Applications
Sample Applications
The PVA ISP package includes sample binaries for testing the ISP and reference source code for custom applications. The binaries are available in the evaluation environment or can be built from source.
Available Applications
| Application | Description |
|---|---|
isp_minimal
|
Minimal application for processing a Bayer RAW image using a YAML parameter file. |
isp_minimal_cuda
|
Extended version of isp_minimal that demonstrates CUDA interoperability.
|
isp_app
|
Full application with histogram processing, auto exposure, V4L2 capture, and multi-frame processing. |
isp_minimal
isp_minimal processes a Bayer RAW image using a YAML parameter file and writes the resulting NV12 image to disk.
Usage
Usage: ./isp_minimal --image <path> --resolution <width>x<height> --parameters <yaml> [--output <path>] [--dump-graph <path>] [--no-convolution]
Example
Download the following reference files:
- IMX274 Bayer RAW image
- Reference parameters file:
params.yaml
decompand: input_shift: 4 x_pts: [0, 1107, 3325, 7766, 16652, 34430, 52207, 56648, 65535] y_pts: [0, 1098, 1885, 2501, 3062, 3602, 3914, 3975, 4095] convolution: kernel: [1, 2, 1, 2, 4, 2, 1, 2, 1] shift: 4 black_level: channel00: 16 channel01: 16 channel10: 16 channel11: 16 digital_gain: gain: 1.5 white_balance: u_ref: [0.982, 0.789, 0.668, 0.558, 0.426] v_ref: [0.308, 0.382, 0.445, 0.428, 0.678] use_calibration: false manual_wb: false manual_wb_gains: [1.0, 1.0, 1.0] demosaic: bayer_pattern: 0 histogram: bit_depth: 16 use_mask: false auto_exposure: kp_exposure: 40.0 ki_exposure: 1.0 kd_exposure: 8.0 kp_gain: 5000000.0 ki_gain: 10.0 kd_gain: 500.0 reference_mean: 120.0 dynamic_range: 255.0 min_exposure: 5000 max_exposure: 20000 min_gain: 2000000 max_gain: 100000000 camera_index: 0 enable_v4l2: false gamma_tone_mapping: gamma: 2.2 global_tone_mapping: brightness: 0.18 gtm_scale_min: 0.0001 gtm_scale_max: 0.01 gtm_temporal_alpha: 1.0 saturation: 1.0 debug: export_scene_key: false export_debug_csv: false debug_export_frame_index: 0
Run:
./isp_minimal \
--image imx274.raw \
--resolution 1920x1080 \
--parameters params.yaml \
--output /tmp/imx274.nv12
Reference results:
- Expected output (visualized with vooya):
- Expected application output log:
isp_minimal output log
[pipeline][debug] Adding stage: decompand0 [pipeline][debug] Adding stage: black_level0 [pipeline][debug] Adding stage: white_balance0 [pipeline][debug] Adding stage: demosaic0 [pipeline][debug] Adding stage: convolution0 [pipeline][debug] Adding stage: global_tone_mapping0 [pipeline][debug] Adding stage: gamma_tone_mapping0 [pipeline][debug] Adding stage: nv12_conversion0 [pipeline][info] Linked decompand0:output -> black_level0:input [pipeline][info] Linked black_level0:output -> white_balance0:input [pipeline][info] Linked white_balance0:output -> demosaic0:input [pipeline][info] Linked demosaic0:output -> convolution0:input [pipeline][info] Linked convolution0:output -> global_tone_mapping0:input [pipeline][info] Linked global_tone_mapping0:output -> gamma_tone_mapping0:input [pipeline][info] Linked gamma_tone_mapping0:output -> nv12_conversion0:input [pipeline][info] Linked black_level0:gains -> white_balance0:gains [pipeline][info] Linked convolution0:scene_key -> global_tone_mapping0:scene_key [pipeline][debug] Registered external frame: BayerRGGB u16 1920x1536 stride=1920 bit_depth=16 size=5898240 [pipeline][debug] Registered external frame: NV12 u8 1920x1536 stride=1920 bit_depth=8 size=4423680 [pipeline][info] Initializing pipeline execution context [pipeline][info] Pipeline initialization complete [pipeline][info] Negotiating pipeline links [pipeline][info] Negotiated decompand0:output -> black_level0:input BayerRGGB u16 1920x1536 stride=1920 bit_depth=16 size=5898240 [pipeline][info] Negotiated black_level0:output -> white_balance0:input BayerRGGB u16 1920x1536 stride=1920 bit_depth=16 size=5898240 [pipeline][info] Negotiated white_balance0:output -> demosaic0:input BayerRGGB u16 1920x1536 stride=1920 bit_depth=16 size=5898240 [pipeline][info] Negotiated demosaic0:output -> convolution0:input RGB u16 1920x1536 stride=1920 bit_depth=16 size=17694720 [pipeline][info] Negotiated convolution0:output -> global_tone_mapping0:input RGB u16 1920x1536 stride=1920 bit_depth=16 size=17694720 [pipeline][info] Negotiated global_tone_mapping0:output -> gamma_tone_mapping0:input RGB u8 1920x1536 stride=1920 bit_depth=8 size=8847360 [pipeline][info] Negotiated gamma_tone_mapping0:output -> nv12_conversion0:input RGB u8 1920x1536 stride=1920 bit_depth=8 size=8847360 [pipeline][info] Negotiated black_level0:gains -> white_balance0:gains Unknown f32 1x3 stride=1 bit_depth=32 size=12 [pipeline][info] Negotiated convolution0:scene_key -> global_tone_mapping0:scene_key Unknown f32 1x2 stride=1 bit_depth=32 size=8 [pipeline][info] Allocating pipeline frames [pipeline][debug] Allocated frame buffer (5898240 bytes) [pipeline][debug] Allocated frame buffer (4423680 bytes) [pipeline][debug] Allocated frame buffer (5898240 bytes) [pipeline][debug] Allocated frame buffer (5898240 bytes) [pipeline][debug] Allocated frame buffer (5898240 bytes) [pipeline][debug] Allocated frame buffer (17694720 bytes) [pipeline][debug] Allocated frame buffer (17694720 bytes) [pipeline][debug] Allocated frame buffer (8847360 bytes) [pipeline][debug] Allocated frame buffer (8847360 bytes) [pipeline][debug] Allocated frame buffer (12 bytes) [pipeline][debug] Allocated frame buffer (8 bytes) [pipeline][info] Configuring pipeline stages [pipeline][info] Configuring stage: decompand0 [pipeline][info] Configuring stage: black_level0 [pipeline][info] Configuring stage: white_balance0 [pipeline][info] Configuring stage: demosaic0 [pipeline][info] Configuring stage: convolution0 [pipeline][info] Configuring stage: global_tone_mapping0 [pipeline][info] Configuring stage: gamma_tone_mapping0 [pipeline][info] Configuring stage: nv12_conversion0 Pipeline initialization time: 33930 us [pipeline][debug] Processing stage: decompand0 [pipeline][debug] Processing stage: black_level0 [pipeline][debug] Processing stage: white_balance0 [pipeline][debug] Processing stage: demosaic0 [pipeline][debug] Processing stage: convolution0 [pipeline][debug] Processing stage: global_tone_mapping0 [pipeline][debug] Processing stage: gamma_tone_mapping0 [pipeline][debug] Processing stage: nv12_conversion0 Pipeline execution time: 9589 us
The source code shows the minimum steps required to load parameters, create the ISP pipeline, process an image, and write the output.
isp_minimal_cuda
isp_minimal_cuda extends isp_minimal and demonstrates interoperability between PVA ISP frames and CUDA.
Its source code can be used as a reference for applications that combine PVA ISP processing with CUDA algorithms.
isp_app
isp_app is the full sample application. It supports raw image files, concatenated raw frames, V4L2 devices, histogram processing, auto exposure, global tone-mapping debug output, and PVA execution statistics.
Usage
Usage: ./isp_app -i <path> -r <res>
[-i | --image] <path>
Path to a raw image or concatenated raw frames file.
Required if not streaming.
[-r | --resolution] <res>
Resolution in the form "<width>x<height>".
Required.
[-d | --device] <dev>
Video device.
Required when --image is not used.
[-n | --num-frames] <frames>
Number of frames to process.
For file input, this may repeat one raw frame or process
multiple concatenated raw frames.
Defaults to 100.
[-f | --format] <pixelfmt>
Capture pixel format:
0: BG12
1: RG10
2: BYR2
Defaults to BG12.
[-p | --parameters] <params>
Path to the YAML file with ISP parameters.
[-m | --mask] <hist-mask>
Path to the float histogram mask.
[-g | --dump-graph] <path>
Write the pipeline graph as a Graphviz DOT file.
[--gtm-luminances-csv] <path>
Output path for the GTM luminances CSV.
Defaults to debug/gtm_luminances.csv.
[--gtm-scene-key-csv] <path>
Output path for the GTM scene-key CSV.
Defaults to debug/gtm_scene_key.csv.
[-s | --stats]
Print PVA execution statistics.
File Input Example
./isp_app \
--image imx274.raw \
--resolution 1920x1080 \
--format 1 \
--parameters params.yaml \
--num-frames 1 \
--stats
V4L2 Input Example
./isp_app \
--device /dev/video0 \
--resolution 1920x1080 \
--format 1 \
--parameters params.yaml \
--num-frames 100 \
--stats
The source code provides a reference for file-based processing, V4L2 capture, histogram integration, auto exposure, and multi-frame ISP pipelines.
Complete Example Source Code
The complete source code for the reference application discussed in this page can be found in the online API documentation
Rather than explaining every line of code individually, this page breaks the application into logical blocks and explains the role of each section.
Introduction
The isp_minimal application demonstrates how to:
- Load a Bayer RAW image.
- Load ISP parameters from a YAML configuration file.
- Create a PVA ISP processing pipeline.
- Configure ISP stages.
- Execute image processing on the PVA.
- Save the resulting NV12 image.
Command-Line Interface
The application begins by parsing command-line arguments and storing them in a configuration structure.
struct AppOptions {
std::string input_path;
std::string output_path;
std::string yaml_path;
std::string graph_path;
uint32_t image_width = 0;
uint32_t image_height = 0;
};
The application expects:
- An input Bayer image.
- An ISP parameter file.
- The image resolution.
- An optional output file path.
- An optional graph export path.
The command-line parsing logic is implemented in the ParseArgs() function.
Loading ISP Parameters
Once the command-line arguments have been validated, the application loads the ISP configuration from a YAML file.
ISPParams params;
std::string load_error;
if (!LoadISPParams(
options.yaml_path,
¶ms,
&load_error)) {
std::cerr
<< "Failed to load ISP params:\n"
<< load_error << "\n";
return 1;
}
The parameter file contains the configuration values used by the ISP algorithms, including:
- Decompanding configuration
- Optical black levels
- White balance gains
- Tone mapping parameters
- Gamma correction values
This allows image tuning without recompiling the application.
Creating the Pipeline
The application creates a Pipeline object that will own and execute the processing graph.
isp::Pipeline pipeline;
isp_example::ConfigurePipelineLogger(
&pipeline);
The pipeline is responsible for:
- Managing stages
- Negotiating frame formats
- Allocating memory
- Executing PVA workloads
Creating Frames
Input and output image buffers are represented by Frame objects.
The example creates a Bayer RAW input frame and an NV12 output frame.
auto* frame_bayer_in =
pipeline->addFrame(
{options.image_width,
options.image_height,
isp::FrameFormat::kBayer,
isp::FrameDataType::kUInt16,
options.image_width,
16,
resources->bayer_size});
auto* frame_nv12_out =
pipeline->addFrame(
{options.image_width,
options.image_height +
(options.image_height / 2),
isp::FrameFormat::kNV12,
isp::FrameDataType::kUInt8,
options.image_width,
8,
resources->nv12_size});
These frames define the boundaries of the processing graph.
Creating ISP Stages
Each ISP algorithm is instantiated independently and added to the pipeline.
auto* dec =
pipeline->add(
isp::createStage(
isp::StageType::kDecompand));
auto* blc =
pipeline->add(
isp::createStage(
isp::StageType::kBlackLevel));
auto* awb =
pipeline->add(
isp::createStage(
isp::StageType::kWhiteBalance));
auto* dmc =
pipeline->add(
isp::createStage(
isp::StageType::kDemosaic));
auto* global_tm =
pipeline->add(
isp::createStage(
isp::StageType::kGlobalToneMapping));
auto* gamma_tm =
pipeline->add(
isp::createStage(
isp::StageType::kGammaToneMapping));
auto* nv12 =
pipeline->add(
isp::createStage(
isp::StageType::kNV12Conversion));
The example builds a complete ISP pipeline using all currently supported image processing stages.
Configuring ISP Algorithms
After creating the stages, the application configures their properties.
For example, the Decompand stage receives its lookup table configuration:
pipeline->setStageProperty(
"decompand0",
"shift_bits",
static_cast<int64_t>(
params.decompand.input_shift));
pipeline->setStageProperty(
"decompand0",
"x_pts",
isp::StagePropertyValue{
params.decompand.x_pts});
pipeline->setStageProperty(
"decompand0",
"y_pts",
isp::StagePropertyValue{
params.decompand.y_pts});
Similar configuration calls are used to configure:
- Black Level Correction
- White Balance
- Global Tone Mapping
- Gamma Tone Mapping
The values are loaded from the YAML parameter file and applied before execution.
Connecting the Pipeline
Once all stages have been created and configured, they are connected together to form the final processing graph.
pipeline->link(dec, blc);
pipeline->link(blc, awb);
pipeline->link(awb, dmc);
pipeline->link(
dmc,
"scene_key",
global_tm,
"scene_key");
pipeline->link(dmc, global_tm);
pipeline->link(global_tm, gamma_tm);
pipeline->link(gamma_tm, nv12);
Notice that some stages exchange additional control information through auxiliary pads, such as the scene_key connection used by the Global Tone Mapping stage.
Pipeline Initialization
Before execution, the graph must be initialized and configured.
pipeline->prepare(bool);
That is equivalent to manually calling these set up steps sequentially:
pipeline->init(bool); pipeline->negotiate(); pipeline->allocate(); pipeline->configure();
These steps perform:
- Graph validation
- Frame negotiation
- Memory allocation
- Stage configuration
- PVA runtime preparation
After successful initialization, the pipeline is ready for execution.
Loading the Input Image
The example reads the Bayer RAW image from disk and copies it into the pipeline input frame.
std::vector<uint16_t> input;
if (!isp_example::LoadRawImageU16(
options.input_path,
resources.bayer_size,
&input)) {
return 1;
}
auto* bayer_input =
reinterpret_cast<uint16_t*>(
resources.frame_bayer_in->hostPtr);
std::memcpy(
bayer_input,
input.data(),
resources.bayer_size);
At this point the pipeline contains all information required to begin processing.
Executing the Pipeline
The entire ISP graph is executed through a single call.
const uint8_t* output = pipeline.process();
This call executes every ISP stage in sequence and returns the processed NV12 image.
The application also measures execution time for performance evaluation using std::chrono.
Writing the Output
The resulting NV12 image is written to disk.
return isp_example::WriteBinaryFile(
options.output_path,
output,
resources.nv12_size)
? 0
: 1;
The generated output can then be:
- Visualized
- Encoded
- Streamed
- Used for AI inference
- Used for image quality evaluation
After understanding the reference application, developers can:
- Build custom ISP pipelines.
- Add or remove ISP stages.
- Integrate PVA ISP into larger applications.
- Create application-specific image processing workflows.
