View source for RidgeRun Video Stabilization Library/Library Integration for IMU/Video Stabilizer Element

<noinclude>
{{RidgeRun Video Stabilization Library/Head|previous=Library_Integration_for_IMU/Add_Support_for_IMU_Sensors|next=Library_Integration_for_IMU/Example_Pipelines|metakeywords=imu,stabilizer,rb5,jetson,nvidia,xilinx,amd,qualcomm}}
</noinclude>


{{DISPLAYTITLE:GStreamer / Video Stabilizer Element
|noerror}}

== Introduction ==

To ease the usage of the RidgeRun Video Stabilization library, a GStreamer plug-in is included to perform accelerated video stabilization based on the IMU readings. The stabilization process takes into account data coming from the sensor, which is matched with the timestamp of the frame to stabilize.

Thanks to this feature, the RidgeRun Video Stabilization GStreamer plug-in does not need buffering and reduces the latency of the stabilization library up to the processing time, which is usually less or equal to the frame interval (1/fps). For instance, if the framerate is 30 fps, the stabilization usually adds 33.3 ms or less.

This wiki will cover the features, properties and supported formats.

== Enabling the Plug-in ==

To enable the plug-in, make sure to have compiled the GStreamer support in the library. It is possible to achieve this with <code>-Denable-gstreamer=enabled</code>. By default, it should have been enabled (see [[RidgeRun_Video_Stabilization_Library/Getting_Started/Building_the_Library#Building_the_project|Building the Project]]).

After the installation, please, check the element by using:

<syntaxhighlight lang=bash>
gst-inspect-1.0 rvstabilize
</syntaxhighlight>

It should show the inspect with details about the formats, properties, signals, and others.

== Supported Formats ==

The supported formats on GStreamer are:

* '''Input''': RGBA
* '''Output''': RGBA

The image dimensions are not constrained. However, it is highly recommended that the dimensions match a multiple of eight (8).

== Features ==

The GStreamer plug-in included in the RidgeRun Video Stabilization library allows the selection of a sensor, fisheye lens undistortion, and hardware acceleration by setting only the element's properties. Here is a list of features:

* '''Sensor Support''': choose from the [[RidgeRun_Video_Stabilization_Library/Getting_Started/Supported_Platforms_and_Backends#Supported_Sensors|list of supported sensors]] provided by the library.
* '''Hardware Acceleration''': choose between OpenCV (CPU), OpenCL, and CUDA with a single selector.
* '''Lens Correction''': camera calibration and undistortion are crucial for good quality results. Pass the camera matrix and distortion coefficients easily without coding.
* '''Variety of Algorithms''': the RidgeRun Video Stabilization includes various algorithms for stabilization. Use them by just selecting them through a property.

When a sensor is not available in the library, the sensor has to be added to the library.

== Properties ==
* '''imu-sensor''': enumerator based on rvs::Sensors. It allows for selecting a sensor in case there are multiple sensors in the same system. For example: <code>bmi160</code>
* '''imu-frequency''': sample rate of the IMU sensor in Hz (unsigned integer). For example: <code>300</code> Hz
* '''imu-id''': sensor ID with unsigned integer. This is when multiple IMU are allowed. For example: <code>0</code>
* '''imu-sensor-device''': describes the path of the sensor in case of I2C sensors. For example: <code>/dev/i2c-7</code>
* '''imu-max-samples''': describes the maximum permitted size for the number of IMU samples in the sensor payload vector (unsigned integer). For example: <code>50</code>
* '''imu-data-order''': string specifying the order of data. The assumption is that the data is XYZ. When an uppercase letter, it means that the data is positive. If lowercase, it is negative. For instance, YxZ means that the Y from the sensor actually represents the positive X of the world, x from the sensor is inverted and it is actually the Y of the sensor. Check [[RidgeRun_Video_Stabilization_Library/Library_Integration_for_IMU/Preparing_the_IMU_Measurements|this wiki]] for reference.

* '''undistort''': enumerator based on rvs::UndistortAlgorithms. It allows the selection of accelerated versions of the undistort, like CUDA, OpenCL or OpenCV. Example: <code>opencv</code>
* '''undistort-fov-scale:''' field-of-view scale with respect to the output. If the number is greater, more zoom out. For example </code>0.7</code>
* '''undistort-intrinsic-matrix''': array of nine elements with the intrinsic camera matrix. Example: <code><2160., 0., 1920., 0., 2160., 1080., 0., 0., 1.}></code>
* '''undistort-distortion-coefficients''': array of four elements with the distortion parameters for fisheye lenses. Example: <code><0.1538203, 0.090335565, 0.01859281, 0.01526001></code>

* '''stabilizer''': enumerator based on rvs::StabilizerAlgorithms. It allows choosing among different stabilization techniques. Example: <code>spherical-exponential-algorithm</code>
* '''stabilizer-horizon-angle''': horizon lock angle in degrees (float). Example: <code>90</code>
* '''stabilizer-smooth-constant''': time constant of the filter (float). Example: <code>0.201</code>

* '''interpolator''': enumerator based on rvs::InterpolatorAlgorithms. It allows choosing among interpolation techniques. Example: <code>slerp</code>
* '''integrator''': enumerator based on rvs::IntegratorAlgorithms. It allows choosing among integration techniques. Example: <code>complementary-integrator-algorithm</code>
* '''hw-device-index''': indicates the device that the backend should use from the platform (unsigned integer). Only valid for OpenCL and CUDA. For example: <code>1</code>
* '''hw-platform-index''': indicates the platform that the backend should use from the system (unsigned integer). Only valid for OpenCL. For example: <code>1</code>

All the properties match the RidgeRun Video Stabilization library options. Please, consider reading [[RidgeRun_Video_Stabilization_Library/Library_Integration_for_IMU|Library Integration for IMU]].

== Missing Something? ==

You need another format, memory type, sensor or platform? Let's talk and add them together: [https://www.ridgerun.com/contact Contact Us].

<noinclude>
{{RidgeRun Video Stabilization Library/Foot|previous=Library_Integration_for_IMU/Add_Support_for_IMU_Sensors|next=Library_Integration_for_IMU/Example_Pipelines}}
</noinclude>