RawBayerToYoloTensorOp

RawBayerToYoloTensorOp fuses RG10 Bayer demosaicing, RGB conversion, aspect-ratio-preserving resize, letterbox padding and YOLO normalization. The operator is optimized for the Holoscan 5 RidgeRun YOLO11 pipeline and emits the TensorRT input directly.

CUDA preprocessing

Contact RidgeRun for custom CUDA preprocessing that connects your raw camera data to your model.

Input arguments

Argument	Default and meaning
`source_width`, `source_height`	Positive, even dimensions of the RG10 source tensor.
`network_width`, `network_height`	`640 × 640` by default; output model dimensions.
`optical_black`	`0`; black-level correction before normalization.
`bayer_grid`	`1` for RGGB; values 0–3 select the supported Bayer phases.
`raw_shift`	`6` by default; right shift before the 10-bit mask.
`input_gain`	`8.0` by default; scales the corrected signal before clamping to `[0,1]`.

Ports and data contract

Port	Memory	Queue	Trigger	Contract
`input`	CUDA device, contiguous	SDK default	`OnEach`	U16 tensor with shape `[source_height, source_width, 1]`.
`output`	CUDA device, runtime pool	N/A	Same activation	New F32 tensor with shape `[1, 3, network_height, network_width]` in planar RGB order.

The operator requires a CUDA stream and forwards the input sample metadata to the output.

Processing behavior

The scale preserves the source aspect ratio. The resized image is centered in the network canvas and the surrounding letterbox pixels are set to zero. For each output pixel, the kernel reads a 3×3 Bayer neighborhood once, estimates RGB values, subtracts the optical-black level, applies input_gain / (1023 - optical_black) and clamps every channel to [0,1].

V4L2UserPtrSourceOp -> RawBayerToYoloTensorOp -> TensorRtInferenceOp -> FakesinkOp

This stage does not decode detections, apply non-maximum suppression, draw boxes or display the image. Those are separate application concerns.

Build

Follow Holoscan RidgeRun Build and Installation for the module build and installation. The operator-specific include, target, and runtime requirements are listed below.

Include raw_bayer_to_yolo_tensor_optimized/raw_bayer_to_yolo_tensor.hpp and link holoscan::raw_bayer_to_yolo_tensor_optimized.

Example

The output device must be bound explicitly when the graph is compiled. The network dimensions should match the TensorRT engine input:

auto preprocess = graph.op<RawBayerToYoloTensorOp>(
    "preprocess", 1920, 1080, 640, 640, 0, 1, 6, 8.0F);
auto inference = graph.op<TensorRtInferenceOp>("inference", model_path);
graph.add_flow(preprocess->output, inference->input);

compile_options.deployment.bind_tensor_output_device(
    holoscan::TensorOutputDevicePlacement{
        .operator_path = "preprocess",
        .output_port = "output",
        .device = holoscan::DeviceId{0}});

The constructor computes the letterbox geometry once. It chooses the smaller network/source scale, rounds the resized dimensions, and centers the image with padding_left and padding_top. Pixels outside the resized image remain zero. Inside the image, the CUDA kernel samples a Bayer neighborhood, produces RGB values, applies the optical-black correction and input_gain / (1023 - optical_black), and writes planar NCHW float32 values.

The operator validates even source dimensions, Bayer phase, RAW10 shift, optical-black range and positive finite gain. At runtime it requires a contiguous CUDA uint16 tensor with the exact configured [height,width,1] shape and a CUDA stream. It forwards sample metadata to the model input and records the preprocessing stage in the RidgeRun flow trace.

Input representation

The operator expects unpacked 16-bit elements containing RG10 values. It does not accept a packed five-byte-per-four-pixel MIPI RAW10 buffer or a host tensor.

Source reference

You can find the source code at RawBayerToYoloTensorOp.

Related pages

V4L2UserPtrSourceOp explains the capture memory lifetime.
TensorRtInferenceOp documents the following model execution stage.
NonMaximumSuppressionOp is an SDK-owned postprocessing option.

❯