Vision Pipeline Example

Follow three synthetic frames through image preprocessing and a separate detection branch. The pipeline makes changes in image shape, data type, layout, and candidate selection visible without requiring a camera, model, or display.

The example is a useful starting point for connecting the Core Vision operators in an application. The Ports, Samples, and Payload Types and Tensor Contracts and Memory Types pages provide the vocabulary used below.

Prerequisites

Use the examples-enabled SDK installation from Build the SDK on AGX Orin. Enable the Vision and examples components during that build. A usable CUDA device is required when the application runs; a missing device produces the documented skip result.

What you will learn

How several operators can form one image-processing path.
How a static Tensor can be published once and reused by later activations.
How a second branch can align boxes and scores and produce an NMS mask.
How an example can check both Tensor values and sample metadata.

Build and run

The shared SDK build page contains the configure and installation steps. After that build, run the installed application in the SDK container:

WORKSPACE="$HOME/holoscan-5.0"
cd "$WORKSPACE/holoscan-sdk"
INSTALL_DIR="$(./run get_install_dir)"
RUN_BIN="/workspace/holoscan-sdk/$INSTALL_DIR/lib/holoscan/examples/vision_pipeline/cpp/vision_pipeline"
./run launch "$INSTALL_DIR" --run-cmd "$RUN_BIN"

The application has no configuration file or command-line options.

Follow the workflow

The source publishes an image, a remap map, boxes, and scores. The image branch applies the following operations:

Operators used

Operator	Result in this example
ResizeOp	Keeps the synthetic U8 image at its original geometry.
CustomCropOp	Selects a central 2 × 2 region.
CopyMakeBorderOp	Adds a zero border around the cropped image.
RemapOp	Samples the central coordinates using a map published with the first frame.
ConvertToOp	Changes the image from U8 to F32.
NormalizeOp	Applies per-channel baseline values.
ReformatOp	Changes the image from interleaved HWC to planar CHW.

The detection branch sends boxes and scores to NonMaximumSuppressionOp. The first frame produces a U8 mask with the expected selection [1,0,1]. Boxes and scores carry matching frame metadata so the operator evaluates the corresponding candidates together.

The image sink checks the final F32 shape [3,2,2], pixel values, frame identifiers, and capture times. The detection sink checks the mask and its metadata. Intermediate operators run on CUDA, while the sinks copy their final results to the host for inspection.

Expected result

A successful run prints:

vision_pipeline: image and detection branches passed

Return code 77 means the example skipped because no usable CUDA device was available. A nonzero result other than 77 indicates a build, graph, observation, or runtime failure. The result validates the programmed values and metadata checks. It is not a latency or copy-free performance measurement.

If testing was enabled, the same application is registered as VISION_PIPELINE_EXAMPLE in CTest:

ctest --test-dir /path/to/holoscan-sdk/build -R '^VISION_PIPELINE_EXAMPLE$' --output-on-failure

Related examples and concepts

Examples

Inference Example carries synthetic device Tensors through a model.
Rate-Aware Ports and Tensor Pool Capacity Example connects rates and storage capacity.

Concepts

Building Graphs and Queue Policies explains the connections and queue choices.
Temporal Contracts and Input Selection explains aligned inputs and reused values.
Zero-Copy Data Flow and CUDA Stream Ordering explains CUDA ordering and readiness.

Source The complete example is in the Vision Pipeline example. The application source contains the graph and checks, and the build definition registers the installed target and test.

Official documentation

NVIDIA Holoscan SDK official documentation provides the current application and operator reference.

❯