Integration for DIS Example Application
 | This page page is under construction. |
Introduction
RidgeRun's Video Stabilization Library provides two Digital Image Stabilization (DIS) example applications that show how to integrate the DigitalStabilizer C++ API in offline and online workflows. These examples estimate motion directly from the video frames using optical flow, smooth the resulting homographies over a temporal window, and apply a geometric transform to produce stabilized output.
The examples covered in this page are:
dis-offline-example: processes a video file or camera source with OpenCV and writes or displays the stabilized output.dis-online-example: captures live frames with GStreamer, stabilizes each frame in application code, and pushes the stabilized buffers to a display pipeline.
These examples use the library API directly. The online example uses GStreamer pipelines for capture and display, but it does not use the rvsdigital GStreamer element.
Offline Digital Video Stabilization
The offline DIS example reads frames from a video source, converts each frame to RGBA, wraps the frame memory as an RVS image, applies DigitalStabilizer, and then writes or displays the stabilized frame.
This example is useful when the input is already available as a file, when testing different stabilization parameters, or when comparing OpenCV and CUDA backends on the same footage.
Instances
This application uses the following main components:
- An OpenCV
VideoCaptureinstance to read the input source. - An optional OpenCV
VideoWriterinstance to save the stabilized output. - A
DigitalStabilizerinstance configured with Gaussian smoothing. OpticalFlowParamsto tune Pyramid Lucas-Kanade feature tracking.- OpenCV or CUDA runtime resources, depending on the selected backend.
- RVS RGBA image wrappers for input and output frames.
Preprocessing and Preparation
The application starts by parsing the command-line options. The input is required and can be:
- A video file path.
- A numeric camera index.
- A V4L2 URI such as
device:///v4l2?index=0.
The example then opens the source with OpenCV. If the example is running in record mode, it creates an MP4 output file. If no output path is provided in record mode, the application creates a default output name based on the input filename.
The example builds DigitalStabilizerParams from the shared DIS options. The configuration uses:
- Pyramid Lucas-Kanade optical flow.
- Gaussian smoothing for the homography window.
- OpenCV or CUDA transform backend, depending on
--backend. - Optional half-resolution optical-flow analysis when
--analysis-mode half-resolutionis selected.
For the CUDA backend, the example creates CUDA runtime settings and persistent CUDA/pinned frame resources. For the OpenCV backend, the example uses host-backed image wrappers.
Main Operation Loop
The main loop processes one frame at a time.
Frame Acquisition
The example reads the next BGR frame from OpenCV. If --max-frames was provided, the loop stops after the requested number of frames.
Frame Preparation
The input frame is converted from BGR to RGBA. The RGBA data is wrapped as an RVS image. The example also prepares an output RVS image using the memory and allocator required by the selected backend.
Digital Stabilization
For each frame, the application calls DigitalStabilizer::Apply() with the backend-specific allocator type. Internally, the stabilizer:
- Estimates frame-to-frame motion with optical flow.
- Pushes the homography into its internal temporal window.
- Smooths the homography sequence.
- Applies the stabilized transform to a delayed frame.
The example checks the returned RuntimeError after each call.
Output
After stabilization, the output frame is converted from RGBA back to BGR. In display mode, the example shows a side-by-side preview of the original and stabilized frames. In record mode, it writes the stabilized frame to the MP4 output file.
The application also prints latency and FPS reports while frames are being processed.
Additional Details
The application is built when examples are enabled and OpenCV support is available. Make sure to configure the project with examples enabled:
meson setup builddir -Denable-examples=enabled -Denable-opencv=enabled
CUDA backend support requires CUDA to be enabled in the build:
meson setup builddir -Denable-examples=enabled -Denable-opencv=enabled -Denable-cuda=enabled
Example usage with the OpenCV backend:
./builddir/examples/concept/dis-offline-example \ --input sample.mp4 \ --output stabilized_output.mp4 \ --backend opencv
Example usage with the CUDA backend:
./builddir/examples/concept/dis-offline-example \ --input sample.mp4 \ --output stabilized_output_cuda.mp4 \ --backend cuda
Example usage with a camera source and side-by-side display:
./builddir/examples/concept/dis-offline-example \ --input "device:///v4l2?index=0" \ --display
Command-line options:
Usage: dis-offline-example [options]
Options:
-h, --help Show this help
-i, --input URI Input URI (required)
-s, --source URI Alias of --input
-o, --output PATH Output MP4 video (record mode)
-n N Number of smoothing frames in [1,9] (default: 5)
--crop C Crop margin proportion (<0.5, default: 0.07)
--backend B Stabilization backend:
cuda
opencv
--homography Enable homography stabilization mode
--harris-refresh N Feature redetection interval (default: 10)
--max-corners N Maximum feature corners (default: 1000)
--pyramid-levels N PyrLK pyramid levels (default: 5)
--analysis-mode M Optical-flow analysis mode:
full-resolution (default)
half-resolution
--max-frames N Process only the first N frames (default: all)
--display Show side-by-side preview (original | stabilized)
Source examples:
--input samples/input.mp4
--input "device:///v4l2?index=0"
Concept: Online Digital Video Stabilization with Live Capture
The online DIS example demonstrates how to integrate DigitalStabilizer in a live application that uses GStreamer for capture and display. The application is split into two independent GStreamer pipelines:
- A capture pipeline that owns the camera and delivers RGBA frames through
appsink. - A display pipeline that receives stabilized RGBA buffers through
appsrc.
Application code sits between both pipelines. It receives each captured buffer, wraps the mapped memory as an RVS image, calls DigitalStabilizer::Apply(), and pushes the stabilized buffer into the display pipeline.
The general frame flow is:
v4l2src -> jpegdec -> videoconvert -> appsink
-> App::OnCaptureSample()
-> DigitalStabilizer::Apply()
-> appsrc -> videoconvert -> videosink
The App Class
The App class is the glue layer of the online example. It owns:
- The capture pipeline helper.
- The display pipeline helper.
- The
DigitalStabilizerinstance. - Backend-specific runtime settings and allocators.
- Optional CUDA-managed buffer pools.
- Per-frame profiling data.
The constructor prepares backend resources and creates the stabilizer. The Setup() method builds the GStreamer pipelines and attaches bus watches. The Start() method starts both pipelines, and Run() enters the GLib main loop.
The Capture Pipeline
The capture helper builds a pipeline that receives an MJPEG stream from a V4L2 camera, decodes it with jpegdec or nvjpegdec, converts it to RGBA, and emits samples through appsink. The example expects the camera to provide JPEG frames at the configured resolution and framerate.
The default capture path is portable and uses regular GStreamer buffers. When --accelerated-gst is enabled, the example uses NVIDIA-oriented elements such as nvjpegdec, nvvidconv, and perf when available.
On GStreamer versions that support appsink allocation negotiation, the CUDA backend can advertise a CUDA-managed buffer pool to the capture pipeline.
OnCaptureSample Callback
When appsink receives a new frame, it invokes the OnCaptureSample callback. This callback:
- Extracts the captured
GstBuffer. - Creates or acquires an output buffer.
- Maps the input and output buffers.
- Wraps each mapped memory span as an RVS RGBA image.
- Calls
DigitalStabilizer::Apply(). - Copies the timing metadata to the stabilized buffer.
- Pushes the stabilized buffer to the display pipeline.
The callback also records latency measurements and stops the application when --max-frames is reached.
The Display Pipeline
The display helper builds a pipeline that receives stabilized RGBA buffers through appsrc and renders them with a video sink.
Backend Paths
The online example supports OpenCV and CUDA backend modes.
In OpenCV mode:
- Input buffers are mapped from regular GStreamer memory.
- Output buffers are allocated with the default GStreamer allocator.
- Both buffers are wrapped as host-backed RVS images.
- The stabilizer runs with OpenCV optical-flow and transform backends.
In CUDA mode:
- CUDA runtime settings are created with an asynchronous CUDA stream.
- The display path uses CUDA-managed buffers when possible.
- The capture path uses CUDA-managed buffers when allocation negotiation is available.
- Fallback device buffers and explicit copies are used when required by the GStreamer version or accelerated GStreamer path.
- The stabilizer runs with CUDA optical-flow and transform backends.
Additional Details
The online example is built when examples, OpenCV, and GStreamer support are enabled:
meson setup builddir -Denable-examples=enabled -Denable-opencv=enabled -Denable-gstreamer=enabled
CUDA support is required to use --backend cuda:
meson setup builddir -Denable-examples=enabled -Denable-opencv=enabled -Denable-gstreamer=enabled -Denable-cuda=enabled
Example usage with the OpenCV backend:
./builddir/examples/concept/dis-online-example \ --device /dev/video0 \ --width 640 \ --height 480 \ --backend opencv
Example usage with the CUDA backend and accelerated GStreamer elements:
./builddir/examples/concept/dis-online-example \ --device /dev/video0 \ --width 1920 \ --height 1080 \ --backend cuda \ --accelerated-gst
Command-line options:
Usage: ./build/examples/concept/dis-online-example [options] Options: -h, --help Show this help -d, --device PATH V4L2 device (default: /dev/video0) --width N Capture/display width (default: 3840) --height N Capture/display height (default: 2160) --fps N Framerate numerator (default: 30) --fps-den D Framerate denominator (default: 1) --accelerated-gst Use nvjpegdec/nvvidconv/perf pipelines --backend B opencv or cuda (default: opencv) --buffers N RGBA buffers per pool when using CUDA (default: 4) -n N Number of smoothing frames in [1,9] (default: 5) --crop C Crop margin proportion (<0.5, default: 0.07) --harris-refresh N Feature redetection interval (default: 10) --max-corners N Maximum feature corners (default: 1000) --pyramid-levels N PyrLK pyramid levels (default: 5) --analysis-mode M full-resolution (default) or half-resolution --max-frames N Process only the first N frames (default: all)
