GStreamer / Video Stabilizer Element

< >

RidgeRun Video Stabilization Library

Table of Contents [Sticky]

• Basics and Foundation
  • What is Video Stabilization
  • Video Stabilization Process using IMU
  • Stabilization Algorithms with IMU
• Getting Started
  • Supported Platforms, Sensors and Backends
  • Evaluating the Library
  • Getting the Code
  • Building the Library
• API Reference
  • Library Architecture
  • Adding New Sensors
  • Adding Algorithms
  • Adding Stabilization Algorithm
  • Adding Backends
  • API Documentation
  • Examples Guidebook
• Library Integration for IMU
  • Preparing the Video
  • Preparing the IMU Measurements
  • Measurement Integration
  • Interpolation
  • Computing the Stabilization
  • Video Undistortion
  • Example Application
• GStreamer
  • Add Support for Video Timestamps
  • Add Support for IMU Sensors
  • Video Stabilizer Element
  • Example Pipelines
• Useful Links
  • BMI160 Setup
  • Performance
• Contact Us

Introduction

To ease the usage of the RidgeRun Video Stabilization library, a GStreamer plug-in is included to perform accelerated video stabilization based on the IMU readings. The stabilization process takes into account data coming from the sensor, which is matched with the timestamp of the frame to stabilize.

Thanks to this feature, the RidgeRun Video Stabilization GStreamer plug-in does not need buffering and reduces the latency of the stabilization library up to the processing time, which is usually less or equal to the frame interval (1/fps). For instance, if the framerate is 30 fps, the stabilization usually adds 33.3 ms or less.

This wiki will cover the features, properties and supported formats.

Enabling the Plug-in

To enable the plug-in, make sure to have compiled the GStreamer support in the library. It is possible to achieve this with -Denable-gstreamer=enabled. By default, it should have been enabled (see Building the Project).

After the installation, please, check the element by using:

gst-inspect-1.0 rvstabilize

It should show the inspect with details about the formats, properties, signals, and others.

Supported Formats

The supported formats on GStreamer are:

• Input: RGBA
• Output: RGBA

The image dimensions are not constrained. However, it is highly recommended that the dimensions match a multiple of eight (8).

Features

The GStreamer plug-in included in the RidgeRun Video Stabilization library allows the selection of a sensor, fisheye lens undistortion, and hardware acceleration by setting only the element's properties. Here is a list of features:

• Sensor Support: choose from the list of supported sensors provided by the library.
• Hardware Acceleration: choose between OpenCV (CPU), OpenCL, and CUDA with a single selector.
• Lens Correction: camera calibration and undistortion are crucial for good quality results. Pass the camera matrix and distortion coefficients easily without coding.
• Variety of Algorithms: the RidgeRun Video Stabilization includes various algorithms for stabilization. Use them by just selecting them through a property.

When a sensor is not available in the library, the sensor has to be added to the library.

Properties

• imu-sensor: enumerator based on rvs::Sensors. It allows for selecting a sensor in case there are multiple sensors in the same system. For example: bmi160
• imu-frequency: sample rate of the IMU sensor in Hz (unsigned integer). For example: 300 Hz
• imu-id: sensor ID with unsigned integer. This is when multiple IMU are allowed. For example: 0
• imu-sensor-device: describes the path of the sensor in case of I2C sensors. For example: /dev/i2c-7
• imu-max-samples: describes the maximum permitted size for the number of IMU samples in the sensor payload vector (unsigned integer). For example: 50
• imu-data-order: string specifying the order of data. The assumption is that the data is XYZ. When an uppercase letter, it means that the data is positive. If lowercase, it is negative. For instance, YxZ means that the Y from the sensor actually represents the positive X of the world, x from the sensor is inverted and it is actually the Y of the sensor. Check this wiki for reference.

• undistort: enumerator based on rvs::UndistortAlgorithms. It allows the selection of accelerated versions of the undistort, like CUDA, OpenCL or OpenCV. Example: opencv
• undistort-fov-scale: field-of-view scale with respect to the output. If the number is greater, more zoom out. For example 0.7
• undistort-intrinsic-matrix: array of nine elements with the intrinsic camera matrix. Example: <2160., 0., 1920., 0., 2160., 1080., 0., 0., 1.}>
• undistort-distortion-coefficients: array of four elements with the distortion parameters for fisheye lenses. Example: <0.1538203, 0.090335565, 0.01859281, 0.01526001>

• stabilizer: enumerator based on rvs::StabilizerAlgorithms. It allows choosing among different stabilization techniques. Example: spherical-exponential-algorithm
• stabilizer-horizon-angle: horizon lock angle in degrees (float). Example: 90
• stabilizer-smooth-constant: time constant of the filter (float). Example: 0.201

• interpolator: enumerator based on rvs::InterpolatorAlgorithms. It allows choosing among interpolation techniques. Example: slerp
• integrator: enumerator based on rvs::IntegratorAlgorithms. It allows choosing among integration techniques. Example: complementary-integrator-algorithm
• hw-device-index: indicates the device that the backend should use from the platform (unsigned integer). Only valid for OpenCL and CUDA. For example: 1
• hw-platform-index: indicates the platform that the backend should use from the system (unsigned integer). Only valid for OpenCL. For example: 1

All the properties match the RidgeRun Video Stabilization library options. Please, consider reading Library Integration for IMU.

Missing Something?

You need another format, memory type, sensor or platform? Let's talk and add them together: Contact Us.

< >