GStreamer DispTEC Tracker Plugin
 |
|
|
GstDispTEC Object Tracking Overview
GstDispTEC object tracker is a GStreamer element which integrates DispTEC [1] library's algorithms for object tracking, making it possible to incorporate their functionalities into GStreamer pipelines, such as showing where the object is located on a frame. This allows the user to strengthen image analysis by connecting its own applications to the pipeline for receiving the analysis data and much more amazing things, as shown in the following demo.
Object Tracking Description
Object tracking is implemented based on Kernelized Correlation Filters (KCF). Each tracker element is initialized with the coordinates of the object in the current frame, snipping the object from the frame and taking it as a template. Afterward, the tracker element updates the filter every time that there is an object hit, adapting itself to the possible changes which an object might have during the video, making it robust enough to cope with potential perspective changes.
This object tracking technique gives the following features:
- Non-stationary camera resilient.
- Tolerates object scaling, rotation, and transformation.
- Resilient to changes in color and light.
GstDispTEC object tracker goes even further, making it possible to reset the object position at run-time, even if the pipeline is in the Play state. This includes:
- Possibility of tracker cascading.
- Tracker reinitialization at run-time.
- Application integrability.
- Visual feedback of object position overlaid to frames.
- Multi-tracker deployment in the same element (coming soon).
The object tracker element gets some parameters to surround the object to get a template and start tracking. Also, it is possible to enable the visual feedback overlaid on the frame highlighting the current position. These parameters are described below:
| Parameter | Description |
|---|---|
| x | (Unsigned int) Object x coordinate at the top-left corner in pixels. Range: 0 - 4294967295. Default: 0 |
| y | (Unsigned int) Object y coordinate at the top-left corner in pixels. Range: 0 - 4294967295. Default: 0 |
| width | (Unsigned int) Object width in pixels. Range: 0 - 4294967295. Default: 100 |
| height | (Unsigned int) Object height in pixels. Range: 0 - 4294967295. Default: 100 |
| render-bbox | (Boolean) Render a bounding box that gives a visual representation of where the object is currently located. Default: TRUE |
The tracker element also generates a signal for third-party applications for retrieving the current object position (coordinates), whose coordinates are expressed as integers in the following sequence: x, y, width, and height.
Object Tracker Architecture
GstDispTEC object tracker has been created based on the DispTEC library. However, it is independent of the DispTEC's dependencies and it can be easily integrated whichever library that DispTEC is based on (OpenCV, LTI-lib, etc). This is achieved by an Abstraction layer just on the base of the plug-in, which can adapt it to several DispTEC variations.
By using DispTEC as the base, and joining it with GStreamer, we offer the best of both worlds: the powerful algorithms available in DispTEC and the flexibility of GStreamer to process videos, allowing live-streaming, pipelining, and more.
GstDispTEC Architecture Description illustrates the general architecture of the plug-in and what makes it one of the best solutions for image analysis for GStreamer.
Supported platforms
GstDispTEC object tracker currently supports platforms that are able to run the Open Computer Vision Library (OpenCV). Some of the platforms already tested are:
| Platform | Operating System |
|---|---|
| Nvidia Jetson TX1 | Ubuntu 16.04 64 bits |
| Personal Computer (PC) | Ubuntu 16.04 64 bits |
Both platforms were running OpenCV 3.3.
Getting the code
Please visit RidgeRun Contact if you are interested in purchasing the GstDispTEC object tracker plugin, with full source code.
Building GstDispTEC
Dependencies
Before trying to install, make sure of having the following dependencies already installed on the target platform:
- OpenCV Core >= 3.3
- OpenCV Contribution Modules >= 3.3
- DispTEC >= 0.2
- GStreamer >= 1.8.1
- Autotools
Installing dependencies
DispTEC
sudo apt install cmake git clone https://github.com/RidgeRun/dispTEC_2017 cmake . make sudo make install
OpenCV
For installing OpenCV, please, follow the steps in the link: OpenCV Installation
GStreamer
sudo apt-get install \ gstreamer1.0-x \ libgstreamer1.0-dev \ libgstreamer-plugins-base1.0-dev
Autotools
sudo apt install \ autotools-dev \ autoconf
Installing GstDispTEC Object Tracker Plugin
When you purchase the gst-dttracker plugin, you will get a Gitlab repository with the source code inside. You need it to build it in your system and add it to your GStreamer plug-ins catalogue. So, in a temporary path, clone the repository using:
git clone git://git@gitlab.com/RidgeRun/orders/${CUSTOMER_DIRECTORY}/gst-disptec.git
Where ${CUSTOMER_DIRECTORY} contains the name of your customer directory given after the purchase.
Afterwards, find the path where GStreamer looks for plug-ins and libraries (LIBDIR). Take as a reference to the following table:
| Platform | LIBDIR path |
|---|---|
| PC 32-bits/x86 | /usr/lib/i386-linux-gnu/ |
| PC 64-bits/x86 | /usr/lib/x86_64-linux-gnu/ |
| Nvidia Jetson | /usr/lib/aarch64-linux-gnu/ |
According to the table, define an environment variable which stores the path accordingly to your system. For example, for an x64 PC, it will be:
LIBDIR=/usr/lib/x86_64-linux-gnu/
Then, do the installation using:
./autogen.sh ./configure --libdir $LIBDIR make sudo make install
Test installation
For testing if everything is properly installed, run:
gst-inspect-1.0 dttracker
It should show:
Factory Details:
Rank none (0)
Long-name DispTEC tracker element
Klass Generic
Description DispTEC object tracking element over a video stream
Author <luis.leon@ridgerun.com, carolina.trejos@ridgerun.com>
Plugin Details:
Name disptec
Description DispTEC Image processing library plugin
Filename /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstdisptec.so
Version 0.3.0
License Proprietary
Source module gst-disptec
Binary package RidgeRun
Origin URL https://www.ridgerun.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseTransform
+----GstVideoFilter
+----GstDttracker
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: BGR
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: BGR
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element Flags:
no flags set
Element Implementation:
Has change_state() function: gst_element_change_state_func
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "dttracker0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
qos : Handle Quality-of-Service events
flags: readable, writable
Boolean. Default: true
x : Initial left corner X position
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 0
y : Initial left corner Y position
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 0
width : Initial object width
flags: readable, writable
Unsigned Integer. Range: 1 - 4294967295 Default: 100
height : Initial object height
flags: readable, writable
Unsigned Integer. Range: 1 - 4294967295 Default: 100
render-bbox : Enables or disables the bounding box around the tracked object
flags: readable, writable
Boolean. Default: true
Element Signals:
"coordinates" : void user_function (GstElement* object,
guint arg0,
guint arg1,
guint arg2,
guint arg3,
gpointer user_data);
Object Tracking Examples
Example 1: Ball pattern with visual feedback
gst-launch-1.0 videotestsrc pattern=ball ! videoconvert ! dttracker name=tracker x=130 y=90 width=50 height=50 enable=true ! videoconvert ! xvimagesink
For this case, the ball always starts at x=130, y=90 and has a width around 50x50 px.
Example 2: Video loaded from a file
INITIAL_OBJ_POSITION="x=100 y=100 width=100 height=100" gst-launch-1.0 filesrc location=$PATH_TO_FILE ! videoconvert ! dttracker name=tracker $INITIAL_OBJ_POSITION enable=true ! videoconvert ! xvimagesink
Set the object initial position into INITIAL_OBJ_POSITION
Example 3: Capturing from camera
INITIAL_OBJ_POSITION="x=100 y=100 width=100 height=100" gst-launch v4l2src device=$CAPTURE_DEVICE ! $CAPS ! dttracker name=tracker $INITIAL_OBJ_POSITION enable=true ! videoconvert ! xvimagesink
References
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
