RidgeRun Video Stabilization Library/Getting Started/Building the Library

< >

RidgeRun Video Stabilization Library

Table of Contents [Sticky]

• Basics and Foundation
  • What is Video Stabilization
  • Video Stabilization Process using IMU
  • Stabilization Algorithms with IMU
• Getting Started
  • Supported Platforms, Sensors and Backends
  • Evaluating the Library
  • Getting the Code
  • Building the Library
• API Reference
  • Library Architecture
  • Adding New Sensors
  • Adding Algorithms
  • Adding Stabilization Algorithm
  • Adding Backends
  • API Documentation
  • Examples Guidebook
• Library Integration for IMU
  • Preparing the Video
  • Preparing the IMU Measurements
  • Measurement Integration
  • Interpolation
  • Computing the Stabilization
  • Video Undistortion
  • Example Application
• GStreamer
  • Add Support for Video Timestamps
  • Add Support for IMU Sensors
  • Video Stabilizer Element
  • Example Pipelines
• Useful Links
  • BMI160 Setup
  • Performance
• Contact Us

Dependencies

The RidgeRun Video Stabilization Library has the following dependencies for building from source:

Jetson Multimedia API

sudo apt install nvidia-l4t-jetson-multimedia-api

Compulsory

A C++ compiler compatible with C++-17.

If GNUCompiler should be installed, we recommend the GNU Compiler >= 9.x.

Moreover, we use the meson building system. You can install it using:

# Install PIP
sudo apt install python3 python3-pip
# Install ninja
sudo apt install ninja-build pkg-config
# Install meson
sudo pip3 install meson

Packages related for building RidgeRun Video Stabilization Library

• GObject bindings for libudev

sudo apt install libgudev-1.0-dev

• V4L2 development package

sudo apt install libv4l-dev

• lbzip2

sudo apt install lbzip2

Semi-Optional

The RidgeRun Video Stabilization Library is built on top of three different backends: OpenCV, OpenCL and CUDA. One of the three must exist in the system. If you are on a Debian-based system, you can install these dependencies using:

OpenCV: For all platforms for CPU execution.

sudo apt install libopencv-core* libopencv-highgui* libopencv-imgproc* libopencv-imgcodecs* libopencv-dev

If the above gives an unmet dependencies error, execute the following command:

sudo apt install libopencv-core-dev libopencv-highgui-dev libopencv-imgproc-dev libopencv-imgcodecs-dev

and then:

sudo apt install libopencv-dev

OpenCL: For non-NVIDIA platforms.

sudo apt install opencl-headers

It may depend on your system

CUDA: For NVIDIA-based platforms (in progress).

sudo apt install cuda-toolkit-*

Optional

For documentation, development and plotting, you can install the following dependencies:

# For documentation
sudo apt install doxygen graphviz
# For plotting
sudo apt install libboost-filesystem-dev libboost-system-dev libboost-iostreams-dev
# For development
pip3 install pre-commit
sudo apt install clang-format cpp-check

In the following sections, you can see how to install the dependencies mentioned before.

GStreamer Support

The RidgeRun Video Stabilization GStreamer element depends on GStreamer Video. You can install it by:

sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good

Building the project

GStreamer only

Everything will be done in the $HOME directory. Let's start by going to home directory.

cd $HOME

1. Installing the GstCameraDriverMeta

• Clone the repo

git clone https://gitlab.ridgerun.com/open/gstreamer/ridgerun-video-stabilization/gstcameradrivermeta.git

• Install

cd gstcameradrivermeta
make
sudo make install

• Return to $HOME directory

cd $HOME

2. Install port for v4l2src GStreamer plug-in

• Clone the repo:

git clone https://gitlab.ridgerun.com/open/gstreamer/gst-v4l2src.git -b feature/add-timestamp-metadata
cd gst-v4l2src

• Configure the build

# Create the folder
mkdir -p ${HOME}/custom-gst-v4l2src

# Configure the path in build, compile and install
meson builddir --prefix=${HOME}/custom-gst-v4l2src --libdir=${HOME}/custom-gst-v4l2src/lib
ninja -C builddir install

• Export the environment variables

export LD_LIBRARY_PATH=${HOME}/custom-gst-v4l2src/lib:${LD_LIBRARY_PATH}
export GST_PLUGIN_PATH=${HOME}/custom-gst-v4l2src/lib/gstreamer-1.0:${GST_PLUGIN_PATH}

These exports can be used in the .bashrc to automate them.

• Test the installation

gst-inspect-1.0 rrv4l2src

It shall show a valid inspect.

• Return to $HOME directory

cd $HOME

3. Install nvarguscamerasrc patched

1. Clone the repo:

git clone https://gitlab.ridgerun.com/open/gstreamer/ridgerun-video-stabilization/gstnvarguscamerasrc
cd gstnvarguscamerasrc

2. Switch to the appropriate branch.

#For JetPack 5.x

git checkout jetpack-5.x

#For JetPack 6.x

git checkout jetpack-6.x

3. Configure the build

# Create the folder
mkdir -p ${HOME}/custom-gst-nvarguscamerasrc/lib/gstreamer-1.0

# Configure the path in build, compile and install
export GST_INSTALL_DIR=${HOME}/custom-gst-nvarguscamerasrc/lib/gstreamer-1.0
make
make install

4. Export the environment variables

export GST_PLUGIN_PATH=${HOME}/custom-gst-nvarguscamerasrc/lib/gstreamer-1.0:${GST_PLUGIN_PATH}

These exports can be used in the .bashrc to automate them.

5. Test the installation

gst-inspect-1.0 nvarguscamerasrc

It shall show a valid inspect. The path of the inspect must match the custom-gst-nvarguscamerasrc

• Return to $HOME directory

cd $HOME

Build RidgeRun Video Stabilization Library

Once the dependencies have been met you can clone the repository. You can run a default compilation with the following:

# When using docs
git submodule update --init

meson build --optimization 3 --prefix /usr -Denable-docs=disabled -Ddeveloper-mode=false

ninja -C build

sudo ninja -C build install

If you encounter this issue:

meson.build:165:4: ERROR: Could not find suitable CUDA compiler: "nvcc"

Run the following:

export PATH=/usr/local/cuda/bin:$PATH

If you encounter the following issue with opencv in Ubuntu 22:

fatal error: opencv2/opencv.hpp: No such file or directory
   19 | #include <opencv2/opencv.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~

Apply the following to the pkgconfig/opencv.pc file. The include path is wrong; it is /usr/include/opencv4/. So, edit the /usr/lib/pkgconfig/opencv4.pc as:

- prefix=/usr/local
+ prefix=/usr

For additional customization, you may refer to the following table of options:

`

Configure Option	Description	Values	Default
-Ddeveloper-mode	Enable developer mode	true, false	true
-Denable-docs	Enable the documentation generation	disabled, enabled	enabled
-Denable-eval	Enable the evaluation system.	disabled, enabled	disabled
-Denable-tests	Enable the compilation of the tests.	disabled, enabled	enabled
-Denable-examples	Enable the compilation of the examples.	disabled, enabled	enabled
-Denable-plots	Enable the compilation of the plotter.	false, true	false
-Denable-opencv	Enable the OpenCV backend compilation (CPU processing)	disabled, enabled	enabled
-Denable-opencl	Enable the OpenCL backend compilation (GPU processing)	disabled, enabled	enabled
-Denable-cuda	Enable the CUDA backend compilation (GPU processing)	disabled, enabled	enabled
-Denable-gstreamer	Enable the GStreamer plug-in	disabled, enabled	enabled
--prefix /usr	Set the installation path of the library	PATH	/usr/local
--optimization 3	Set the optimization level to the maximum	0, 1, 2, 3	0

Table 1. RidgeRun Video Stabilization Library configuration options (Meson)

Note: If the dependency is not found, it will skip the compilation, and a message will be shown during the construction.

If when a GStremear pipeline is run and shows up the following error:

(gst-plugin-scanner:8314): GStreamer-WARNING **: 10:34:29.285: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstrvstabilize.so': libgstcameradrivermeta.so: cannot open shared object file: No such file or directory
No such element or plugin 'rvstabilize'

Apply the following:

sudo mv /usr/local/lib/libgstcameradrivermeta.so /usr/lib/

Testing the project

You can test the final construction results of the library using the test suite:

ninja -C build test

Moreover, if the examples were enabled, you can run them as well: Examples Guidebook

Using the library for your app

A simple application without options, can be compiled using:

g++ -o app app.cpp $(pkg-config rvs --cflags --libs)

where app.cpp is the app to compile and uses the RidgeRun Video Stabilization Library.

You can use pkg-config with the package name of rvs. The library has to be installed previously.

< >