GStreamer / Example Pipelines
 |
|
Introduction
The GStreamer plug-in for the RidgeRun Video Stabilization library aims to be easy to use, accepting various video sources and supporting sensors coming from the library. This section will show some pipelines that can be used as examples.
As a note, the element requires a live source, requiring an actual camera to work. Synthetic video sources like video recordings and test sources will not work for testing.
The following video was generated using our GStreamer element with a Xavier NX and a BMI160:
Example Pipelines
This section contains some examples of the video stabilization library element. To define the properties, please, check the Defining Properties section for details.
Webcams and V4L2-compliant Cameras (non-Bayer cameras)
For V4L2-compliant cameras that produce already demosaic videos in formats like YUYV, I420, RGBA, NV12, etc. It includes webcams.
Output to Display
gst-launch-1.0 rrv4l2src ! \
video/x-raw,width=640,height=480 ! videoconvert ! queue ! \
\
rvstabilize undistort=opencv imu-sensor=bmi160 \
imu-sensor-device=/dev/i2c-8 imu-data-order="zyx" \
stabilizer-smooth-constant=0.2 undistort-fov-scale=0.7 \
undistort-intrinsic-matrix="<845.63f, 0.f, 348.764f, \
0.f, 849.12470406f, 273.80647627f, 0.f, 0.f, 1.f>" \
\
! queue ! videoconvert ! ximagesink sync=true
Output to Recording
gst-launch-1.0 rrv4l2src ! \
video/x-raw,width=640,height=480 ! videoconvert ! queue ! \
\
rvstabilize undistort=opencv imu-sensor=bmi160 \
imu-sensor-device=/dev/i2c-8 imu-data-order="zyx" \
stabilizer-smooth-constant=0.2 undistort-fov-scale=0.7 \
undistort-intrinsic-matrix="<845.63f, 0.f, 348.764f, \
0.f, 849.12470406f, 273.80647627f, 0.f, 0.f, 1.f>" \
\
! queue ! videoconvert ! x264enc ! mp4mux ! \
filesink location=recording.mp4 -e
Tested environment:
- NVIDIA Jetson Xavier NX with Jetpack 5.1.2
- BMI160 Sensor Module Brick over I2C
- Webcam Trust Trino HD
MIPI Cameras Connected to a Jetson
For Argus-compliant cameras whose outputs can be processed by the ISP to get NV12 or I420 outputs.
Output to Display
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! \
video/x-raw,width=1920,height=1080,format=RGBA ! queue ! \
\
rvstabilize undistort=cuda imu-sensor=bmi160 \
imu-sensor-device=/dev/i2c-8 imu-data-order="zyx" \
stabilizer-smooth-constant=0.2 undistort-fov-scale=0.7 \
undistort-intrinsic-matrix="<845.63f, 0.f, 348.764f, \
0.f, 849.12470406f, 273.80647627f, 0.f, 0.f, 1.f>" \
\
! queue ! nveglglessink sync=true
Output to Recording
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! \
video/x-raw,width=1920,height=1080,format=RGBA ! queue ! \
\
rvstabilize undistort=opencv imu-sensor=bmi160 \
imu-sensor-device=/dev/i2c-8 imu-data-order="zyx" \
stabilizer-smooth-constant=0.2 undistort-fov-scale=0.7 \
undistort-intrinsic-matrix="<845.63f, 0.f, 348.764f, \
0.f, 849.12470406f, 273.80647627f, 0.f, 0.f, 1.f>" \
\
! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! mp4mux ! \
filesink location=recording.mp4 -e
Tested environment:
- NVIDIA Jetson Xavier NX with Jetpack 5.1.2
- BMI160 Sensor Module Brick over I2C
- Camera Sony IMX477
Defining Properties
The element is configurable through properties. This section will cover some of the mandatory properties and possible values.
imu-sensor
Sets the model of the sensor connected to the platform:
imu-sensor=bmi160
Possible values:
- bmi160 (default)
- rb5
imu-frequency
Sets the frequency of reading of the sensor:
imu-frequency=200
Possible values:
- Integer number indicating Hertz (Hz)
The recommended frequency is often four times the framerate. Some sensors may be limited by the i2c speed. Usually, the maximum for the BMI160 is between 200-300 Hz.
imu-sensor-device
Sets the path to the device. It depends on the connection with the sensor. For the BMI160, it is I2C.
imu-sensor-device=/dev/i2c-8
The Jetson Xavier NX has the sensor connected to the /dev/i2c-8 if connected to as illustrated in BMI160 Setup.
imu-data-order
Sets the orientation mapping of the axes of the sensor w.r.t. the axes of the image reference system. It consists of a string with three characters. Please, check the Preparing the IMU Measurements for reference.
imu-data-order="zyx"
Default: "XYZ"
undistort
Selects the implementation of the undistort in terms of the hardware acceleration.
undistort=cuda
Possible values:
- opencv (default)
- opencl
- cuda
Check Supported Platforms, Sensors and Backends for reference.
undistort-fov-scale
Scales the field-of-view as a sort of zoom to the image. It multiplies the scale factor by the focal length to create a zoom effect. For example, 0.7 will zoom in the image, whereas 3.0 will zoom out the image.
undistort-fov-scale=0.7
Possible values:
- floating point numbers. Default: 1.5
undistort-intrinsic-matrix
Sets the camera intrinsic matrix for better mapping between the image and world reference systems.
undistort-intrinsic-matrix="<845.62932614f, 0.f, 348.76379855f, 0.f, 849.12470406f, 273.80647627f, 0.f, 0.f, 1.f>"
Possible values:
- array of nine values
stabilizer-smooth-constant
Sets the constant of smoothness of the stabilization filter.
stabilizer-smooth-constant=0.2
Possible values:
- Floating-point numbers between 0 and 1.
Others
There are other properties. Please, check them in Video Stabilizer Element.