Examples - Sample Pipelines

Follow us on:

Share This Page

Facebook Twitter LinkedIn WhatsApp Email Copy Link

Close

Send Email Using

Gmail (Web) Thunderbird (default mail app)

Close

PVA ISP for NVIDIA Jetson

< >

PVA ISP for
NVIDIA Jetson

Table of Contents [Sticky]

• What is PVA ISP?
  • Key Benefits
  • Supported Platforms,Cameras & Formats
  • Alternatives
• Getting Started
  • How to Get the Code
  • Building PVA ISP
  • Evaluating PVA ISP
• User Guide
  • Integration Options
  • Supported ISP algorithms
  • Debugging, Profiling & Tuning
  • Custom Application Development
• Examples
  • Sample Applications
  • Sample Pipelines
  • Demos
• Performance
  • Latency & Throughput
• Contact Us

Sample Pipelines

These pipelines use the pvaisp GStreamer element to process Bayer RAW input and produce NV12 video. For the C++ API examples, see Sample Applications.

The commands below assume 1920 × 1080 BGGR input, with 12-bit samples stored in 16-bit words. They use the params-reference.yaml file supplied with the PVA ISP GStreamer element, whose demosaic.bayer_pattern is 0 (BGGR). Run the commands from a directory containing that file, or replace its path. Change the Bayer caps, resolution, frame rate, and ISP parameters to match your input. Use even image dimensions for NV12 output.

Prerequisites

Install the PVA ISP GStreamer plugin and check its elements:

gst-inspect-1.0 pvaisp
gst-inspect-1.0 pvacudaupload

The camera examples use the separate rrv4l2src element. Check it before running those examples:

gst-inspect-1.0 rrv4l2src
v4l2-ctl --device=/dev/video0 --list-formats-ext

The recording and streaming examples also require nvvidconv and nvv4l2h264enc on a Jetson with H.264 hardware encoding support. Check their availability with gst-inspect-1.0; see NVIDIA's accelerated GStreamer guide for encoder details. The display example requires videoconvert and autovideosink.

Element	Purpose
pvaisp	Processes Bayer input and produces NV12 output. Its sink accepts rggb, bggr, gbrg, or grbg with bpp=10, 12, 14, or 16.
pvacudaupload	Copies upstream Bayer buffers into the output buffers negotiated with pvaisp. Use it when a file source or camera source cannot fill the ISP input buffers directly.
rrv4l2src	Captures Bayer frames from a V4L2 device. Its io-mode selects the camera buffer exchange mode.

Configuring pvaisp

The params property selects a YAML parameter file. It is optional: when omitted, pvaisp uses built-in defaults. A property set on the element overrides the corresponding value in the YAML file. For example, append auto-exposure-reference-mean=50 to a pvaisp element in a pipeline to override that setting.

The use-convolution and use-ccm properties select optional processing stages; both default to false. Set use-ccm=true to enable color correction, use-convolution=true to enable convolution, or both to use the fused convolution and color correction stage. Set these options, params, and session-id before processing starts. Use gst-inspect-1.0 pvaisp to see the available properties.

Keep the caps Bayer pattern consistent with demosaic.bayer_pattern: 0 = BGGR, 1 = RGGB, 2 = GBRG, and 3 = GRBG. The reference file also contains decompanding values for its example input; tune the input shift, decompanding curve, black level, and gains for the camera before evaluating image quality. The reference file has auto_exposure.enable_v4l2: false, so the pipelines below do not write exposure or gain controls to the camera.

Sample Pipelines

Process a Bayer RAW File

This pipeline reads one 1920 × 1080 frame, processes it, and writes one headerless NV12 frame to capture.nv12:

gst-launch-1.0 -e \
    filesrc location=capture.raw blocksize=4147200 num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=1/1' ! \
    pvacudaupload ! \
    pvaisp params=params-reference.yaml ! \
    'video/x-raw,format=NV12,width=1920,height=1080' ! \
    filesink location=capture.nv12

Here blocksize=4147200 is 1920 × 1080 × 2 bytes. For a file containing consecutive frames in the same layout, increase or omit num-buffers; keep blocksize equal to the bytes per input frame. One tightly packed 1920 × 1080 NV12 frame contains 1920 × 1080 × 3 / 2 = 3110400 bytes. The raw NV12 file contains no dimensions or timing metadata.

Capture with V4L2 USERPTR

This path connects rrv4l2src directly to pvaisp. It can avoid an upload copy when the source uses the downstream buffer allocation. Whether capture is copy-free depends on the camera driver and negotiated buffer pool.

gst-launch-1.0 -e \
    rrv4l2src device=/dev/video0 io-mode=userptr num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=30/1' ! \
    pvaisp params=params-reference.yaml ! \
    'video/x-raw,format=NV12,width=1920,height=1080' ! \
    filesink location=capture.nv12

The result is one headerless NV12 frame in capture.nv12. Remove num-buffers=1 for continuous capture.

Capture with V4L2 MMAP

When camera buffers cannot be supplied through the ISP buffer pool, insert pvacudaupload to copy each captured frame before processing:

gst-launch-1.0 -e \
    rrv4l2src device=/dev/video0 io-mode=mmap num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=30/1' ! \
    pvacudaupload ! \
    pvaisp params=params-reference.yaml ! \
    'video/x-raw,format=NV12,width=1920,height=1080' ! \
    filesink location=capture.nv12

Record H.264 in MP4

This pipeline converts the ISP's NV12 output to the NVMM buffers expected by the Jetson H.264 encoder, then writes recording.mp4. Use Ctrl+C to stop; -e lets the muxer finish the file.

gst-launch-1.0 -e \
    rrv4l2src device=/dev/video0 io-mode=userptr ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=30/1' ! \
    pvaisp params=params-reference.yaml ! \
    'video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! \
    nvv4l2h264enc ! h264parse ! qtmux ! \
    filesink location=recording.mp4

Display the ISP Output

This pipeline displays the processed video using the available video sink:

gst-launch-1.0 -e \
    rrv4l2src device=/dev/video0 io-mode=userptr ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=30/1' ! \
    pvaisp params=params-reference.yaml ! \
    'video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1' ! \
    videoconvert ! autovideosink

Stream H.264 over RTP/UDP

Set host to the receiving machine's address. The pipeline sends H.264 in RTP packets to UDP port 5000:

gst-launch-1.0 -e \
    rrv4l2src device=/dev/video0 io-mode=userptr ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=30/1' ! \
    pvaisp params=params-reference.yaml ! \
    'video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1' ! \
    nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! \
    nvv4l2h264enc insert-sps-pps=true ! h264parse ! \
    rtph264pay pt=96 config-interval=1 ! \
    udpsink host=192.168.1.100 port=5000

Share an Execution Session Across Streams

Set the same nonempty session-id on multiple pvaisp elements in one process to share a PVA execution context. Each element keeps its own ISP graph and frame state; processing calls through the shared context are serialized. For independent execution contexts, use different IDs.

gst-launch-1.0 -e \
    filesrc location=input0.raw blocksize=4147200 num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=1/1' ! \
    pvacudaupload ! queue ! pvaisp session-id=pva0 params=params-reference.yaml ! \
    filesink location=output0.nv12 \
    filesrc location=input1.raw blocksize=4147200 num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=1/1' ! \
    pvacudaupload ! queue ! pvaisp session-id=pva0 params=params-reference.yaml ! \
    filesink location=output1.nv12

Troubleshooting

• Check that rrv4l2src, pvaisp, and any other elements in the selected pipeline are installed and can be inspected with gst-inspect-1.0.
• Compare the camera's reported Bayer format, resolution, frame rate, and V4L2 I/O modes with the caps in the pipeline. The samples require Bayer pixels in 16-bit words; packed RAW10 or RAW12 needs unpacking before this path.
• Check the Bayer pattern in both the caps and ISP parameters. An incorrect pattern can produce wrong colors even when caps negotiation succeeds.
• For file input, make blocksize match a complete input frame. Check the capture driver's bytes-per-line and frame size if it adds padding.
• If direct USERPTR capture cannot negotiate buffers with pvaisp, try the MMAP pipeline with pvacudaupload.
• For encoding, verify that nvvidconv produces video/x-raw(memory:NVMM),format=NV12 for the hardware encoder.

Test capture independently of the ISP. Adjust the caps to a format reported by the camera:

gst-launch-1.0 -v rrv4l2src device=/dev/video0 io-mode=mmap num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=10,width=3840,height=2160,framerate=30/1' ! \
    fakesink silent=false

gst-launch-1.0 -v rrv4l2src device=/dev/video0 io-mode=mmap num-buffers=1 ! \
    'video/x-bayer,format=bggr,bpp=12,width=1920,height=1080,framerate=30/1' ! \
    fakesink silent=false

Back To Top ⭡ < >