DigitalStabilizer Class

Table of Contents [Sticky]

Introduction

The DigitalStabilizer class provides an end-to-end digital image stabilization pipeline based on frame-to-frame motion estimation. It receives video frames, estimates visual motion with optical flow, smooths the resulting transformations over a temporal window, and applies a geometric transform to produce a stabilized output frame.

This stabilizer is useful for applications where the stabilization data is extracted from the video stream itself. The pipeline operates on consecutive frames and keeps internal state between calls, so the same stabilizer instance must be reused while processing a continuous stream.

Pipeline Overview

The digital stabilization pipeline is composed of three main stages:

Optical flow estimation: tracks visual features between consecutive frames and estimates a 3x3 homography that represents the apparent frame-to-frame motion.
Transformation smoothing: stores recent homographies in an internal temporal window and computes a smoothed transformation.
Image transformation: applies the smoothed transformation to a delayed frame and writes the stabilized image to the output frame.

The processing flow is:

Configure the digital stabilization parameters.
Create a DigitalStabilizer instance.
Wrap input and output frame memory using RVS image objects.
Call Apply() once per frame.
Check the returned RuntimeError.
Reuse the same stabilizer instance for the next frame.

Parameters

The DigitalStabilizerParams structure configures the complete digital stabilization chain.

Parameter	Description	Default
`smoothing_frames`	Number of frames around the center frame used for temporal smoothing.	`5`
`crop_margin`	Crop margin used by the transform backend to keep valid output bounds after warping.	`0.07`
`optical_flow_algorithm`	Optical-flow backend used to estimate frame-to-frame motion.	`kPyrLKCuda`
`smoothing_algorithm`	Smoothing backend used to smooth the homography window.	`kGaussian`
`transform_algorithm`	Geometric transform backend used to warp the delayed frame.	`kTransformOpenCV`
`optical_flow_params`	Optional optical-flow tuning parameters.	`nullptr`
`smoothing_params`	Optional smoothing tuning parameters.	`nullptr`
`half_resolution`	Optional half-resolution processing mode to reduce compute cost.	`false`

Methods

The class provides the following public methods:

DigitalStabilizer: constructs the stabilizer using optional pipeline parameters, runtime settings, and logger. If no parameters are provided, the default DigitalStabilizerParams values are used.
Apply: stabilizes one input frame and writes the result to the output frame. This method updates the internal delay buffers and motion history. The templated overload also validates that the input and output frames use the expected allocator type.
Reset: clears the internal frame and homography buffers. The stabilizer initializes them again on the next frame.
UpdateParams: replaces the current pipeline parameters and resets the internal state.
SetRuntimeSettings: updates the runtime settings used by the optical-flow and transform backends.
GetRuntimeSettings: returns a copy of the runtime settings currently used by the stabilizer sub-components.
GetParams: returns a copy of the current digital stabilization parameters.

Internal Buffering

Digital stabilization is stateful. The optical-flow stage compares the current frame against previous frame state, while the smoothing stage requires a temporal window of transformations. For that reason, the stabilizer keeps internal frame and homography buffers.

The output produced by Apply() corresponds to a delayed frame from the internal queue. This delay lets the smoothing stage use temporal context around the stabilized frame instead of applying only the newest instantaneous transform. A larger smoothing window generally produces steadier output, but it also increases latency and can make the result less responsive to intentional camera motion.

When the input stream changes, the frame size changes, or the application needs to restart stabilization, call Reset() before processing the next frame.

Runtime Settings

Runtime settings are passed to the stabilizer sub-components that need backend-specific execution data. For example, accelerated backends may require CUDA runtime settings, streams, or other execution context. Applications can provide runtime settings when constructing the stabilizer or update them later with SetRuntimeSettings().

The input and output images must use memory and allocator types compatible with the selected backend. For example, a CPU/OpenCV path typically uses host images, while a CUDA path uses CUDA-compatible image allocation.

Basic Usage

The following example shows the basic structure for creating and using the digital stabilizer. The exact image type and allocator depend on the selected backend.

#include <memory>
#include <rvs/stabilizers/digital.hpp>

auto optical_flow_params = std::make_shared<rvs::OpticalFlowParams>();
optical_flow_params->feature_max_corners = 1000;
optical_flow_params->feature_redetect_interval = 10;
optical_flow_params->opt_flow_num_levels = 2;

auto params = std::make_shared<rvs::DigitalStabilizerParams>();
params->smoothing_frames = 5;
params->crop_margin = 0.07F;
params->optical_flow_algorithm = rvs::OpticalFlowAlgorithms::kPyrLKOpenCV;
params->smoothing_algorithm = rvs::SmoothingAlgorithms::kGaussian;
params->transform_algorithm = rvs::TransformAlgorithms::kTransformOpenCV;
params->optical_flow_params = optical_flow_params;

std::shared_ptr<rvs::IRuntimeSettings> runtime_settings = nullptr;
rvs::DigitalStabilizer stabilizer(params, runtime_settings);

for (;;) {
  std::shared_ptr<rvs::IImage> input_frame = GetNextInputFrame();
  std::shared_ptr<rvs::IImage> output_frame = GetOutputFrame();

  rvs::RuntimeError ret =
      stabilizer.Apply<rvs::HostAllocator>(output_frame, input_frame);
  if (ret.IsError()) {
    break;
  }

  DisplayOrStoreFrame(output_frame);
}

Backend Selection

The digital stabilizer is composed from three backend families:

Optical flow: estimates frame-to-frame motion. Available algorithm selectors include kDISOpenCV, kPyrLKOpenCV, and kPyrLKCuda.
Smoothing: smooths the transformation sequence. The digital pipeline commonly uses kGaussian for homography smoothing.
Transform: applies the final projective transform to the image. Available algorithm selectors include kTransformOpenCV and kTransformCuda.

The selected backends must be available in the current build and compatible with the image memory used by the application.

Error Handling

Most methods return RuntimeError. Applications should check this return value after each call. Common error sources include null input or output frames, incompatible allocators, unavailable backends, invalid parameter values, or incompatible image memory for the selected backend.

Example Applications

The repository includes concept examples that show how to integrate the digital stabilizer in offline and online pipelines:

examples/concept/dis-offline-example.cpp
examples/concept/dis-online-example.cpp

These examples demonstrate parameter configuration, backend selection, frame wrapping, runtime settings, and per-frame calls to Apply().

Reference Documentation

Reference Documentation: DigitalStabilizer

❯

Share This Page

Send Email Using