GStreamer DispTEC Motion Detection Plugin

From RidgeRun Developer Wiki



Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.



GStreamer DispTEC Motion Detection overview

The GstDispTEC motion detection is a GStreamer element which integrates DispTEC [1] library's algorithms for motion detection, making it possible to incorporate their functionality into GStreamer pipelines, such as motion detection in a video sequence, even with non-stationary cameras! This allows the user to strengthen image analysis, by connecting its own applications to the pipeline for receiving the analysis data and much more amazing things, as shown in the following sample video. Also, the algorithms can be run either on CPU or GPU, making the most of the computational resources and reducing the processing time.

Load video
YouTube

Motion Detection Description

The motion detection element is based on DispTEC's MCD and SCBU algorithms, designed to run on both CPU and GPU. Its primary feature is the motion detection inside a non-stationary capture, although it can also be used on stationary captures. When deciding which of the algorithms to use, consider the following guidelines [2]:

  • MCD algorithm is faster, especially for smaller resolutions, and the result contains less noise than its counterpart.
  • SCBU adjusts better to brightness changes on the captured video but it is more sensible to fast changes in movement. The output is more uniform but contains more noise than MCD.

Dtmotion is the designed GStreamer element that integrates the previously described capabilities into a videofilter derived element that can be easily incorporated into a pipeline. It receives any input and generates a new frame built upon a binary mask that exposes the detected motion. In order to adjust the element to the desired application, the following properties can be modified:

Name Description
algorithm Both MCD and SCBU algorithms are available to run on CPU or GPU, depending on the target application. If not specified, the default MCD algorithm on CPU is selected. The available options are the following: 
                          (0): cpu-mcd          - MCD on CPU
                          (1): cpu-scbu         - SCBU on CPU
                          (2): gpu-mcd          - MCD on GPU
                          (3): gpu-scbu         - SCBU On GPU
block-size (Only for SCBU) Unsigned Integer. Range: 0 - 4294967295. Default: 4.

It is an important measure of the algorithm's granularity. The captured image is divided into a number of blocks depending on the specified size. If it is set to be too small, the slightest change between frames may result in motion detection. It will also require greater memory usage, but the processing will be faster and more efficient. Greater value will reduce the amount of noise on the output, and require less memory, but the processing is less efficient.

grid-size (Only for SCBU) Unsigned Integer. Range: 0 - 4294967295 Default: 60

Grid size determines the distance between the resulting amount of points that will be used to determine how much the background moved. The more points are used, the more precise the measurement. However, the algorithm used is fairly resistant to noise, so it won't make much difference to use a smaller value. A large grid size (fewer points) usually yields good results.

max-learning-rate (Only for SCBU) Unsigned Integer. Range: 0 - 4294967295 Default: 5

The varying learning rate property is a key feature if trying to detect motion inside of non-static captures, given that it allows adaptation to object movement vs background movement. Increasing the learning rate decreases the sensibility to object motion detection, but improves the rate at which it adapts to background changes. A low maximum value helps in noisy captures due to the fact that the detected moving objects won't be absorbed as background.

Motion Detection Architecture

GstDispTEC motion detection has been created based on the DispTEC library. However, it is independent of the DispTEC's dependencies and it can be easily integrated whichever library that DispTEC is based on (OpenCV, LTI-lib, etc). This is achieved by an Abstraction layer just on the base of the plug-in, which can adapt it to several DispTEC variations.

By using DispTEC as the base, it is possible to perform and run Computer Vision algorithms easily on CPU/GPU, accelerating the performance and delivery time of video analyses. Nevertheless, joining DispTEC with GStreamer, we offer the best of both worlds: the powerful algorithms available in DispTEC and the flexibility of GStreamer to process videos, allowing live-streaming, pipelining, and more.

GstDispTEC architecture diagram.

GstDispTEC Architecture Description illustrates the general architecture of the plug-in and what makes it one of the best solutions for image analysis for GStreamer.

Supported platforms

The GstDispTEC motion detection plugin currently supports platforms capable to run the Open Computer Vision Library (OpenCV). For a better experience, it is recommended to run on a platform with CUDA support for accelerating video processing. Some of the platforms already tested are:

Platform Operating System
NVIDIA®Jetson™ TX1 Ubuntu 16.04 64 bits
Personal Computer (PC) Ubuntu 16.04 64 bits

Both platforms were running OpenCV 3.3.

In case of not having a platform not capable of running CUDA, gst-dtmotion will run on CPU.

Getting the code

Please contact RidgeRun homepage, if you are interested in purchasing the GstDispTEC motion detector plugin, with full source code. RidgeRun contact details are mentioned in the Contact Us section at the end of this wiki.

Building GstDispTEC

Dependencies

Before trying to install, make sure of having the following dependencies already installed on the target platform:

  • OpenCV Core >= 3.3
  • OpenCV Contribution Modules >= 3.3
  • DispTEC >= 0.2
  • CUDA >= 9.0
  • GStreamer >= 1.8.1
  • Autotools

Installing dependencies

DispTEC 

sudo apt install cmake
git clone https://github.com/RidgeRun/dispTEC_2017
cmake .
make
sudo make install

OpenCV

For installing OpenCV, please, follow the steps in the link: OpenCV Installation

GStreamer 

sudo apt-get install \
gstreamer1.0-x \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev

Autotools 

sudo apt install \
autotools-dev \
autoconf

Installing GstDispTEC Motion Detection

When you purchase the GStreamer motion detection plugin, you will get a Gitlab repository with the source code inside. You need it to build it in your system and add it to your GStreamer plug-ins catalogue. So, in a temporary path, clone the repository using: 

git clone git://git@gitlab.com/RidgeRun/orders/${CUSTOMER_DIRECTORY}/gst-disptec.git

Where ${CUSTOMER_DIRECTORY} contains the name of your customer directory given after the purchase.

Afterwards, find the path where GStreamer looks for plug-ins and libraries (LIBDIR). Take as a reference to the following table:

Platform LIBDIR path
PC 32-bits/x86 /usr/lib/i386-linux-gnu/
PC 64-bits/x86 /usr/lib/x86_64-linux-gnu/
NVIDIA®Jetson™ /usr/lib/aarch64-linux-gnu/

According to the table, define an environment variable which stores the path accordingly to your system. For example, for an x64 PC, it will be: 

LIBDIR=/usr/lib/x86_64-linux-gnu/

Then, do the installation using: 

./autogen.sh
./configure --libdir $LIBDIR
make
sudo make install

Test installation

For testing if everything is properly installed, run: 

gst-inspect-1.0 dtmotion

It should show: 

Factory Details:
  Rank                     none (0)
  Long-name                DispTEC motion detector
  Klass                    Filter/Video
  Description              DispTEC motion detector element over a video stream
  Author                   carolina.trejos@ridgerun.com, luis.leon@ridgerun.com

Plugin Details:
  Name                     disptec
  Description              DispTEC Image processing library plugin
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstdisptec.so
  Version                  0.3.0
  License                  Proprietary
  Source module            gst-disptec
  Binary package           RidgeRun
  Origin URL               https://www.ridgerun.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseTransform
                         +----GstVideoFilter
                               +----GstDtmotion

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: BGR
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: GRAY8
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_element_change_state_func

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "dtmotion0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  qos                 : Handle Quality-of-Service events
                        flags: readable, writable
                        Boolean. Default: true
  algorithm           : Select motion detection algorithm
                        flags: readable, writable
                        Enum "GstDtmotionAlgorithm" Default: 0, "cpu-mcd"
                           (0): cpu-mcd          - MCD on CPU
                           (1): cpu-scbu         - SCBU on CPU
                           (2): gpu-mcd          - MCD on GPU
                           (3): gpu-scbu         - SCBU On GPU
  block-size          : Block size
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4 
  grid-size           : Grid size
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 60 
  max-learning-rate   : Maximum learning rate
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 5

Examples

Example 1: Default settings 

gst-launch-1.0 filesrc location=$PATH_TO_FILE ! decodebin name=dec ! videoconvert ! dtmotion algorithm=cpu-mcd ! videoconvert ! xvimagesink

Example 2: Setting all properties 

gst-launch-1.0 filesrc location=$PATH_TO_FILE ! decodebin name=dec ! videoconvert ! dtmotion algorithm=cpu-scbu max-learning-rate=5 block-size=4 grid-size=60 ! videoconvert ! xvimagesink

Example 3: Capturing from camera 

CAPTURE_DEVICE=/dev/video0
CAPS='video/x-raw, format=(string)UYVY, width=(int)640, height=(int)360, framerate=30/1'
gst-launch-1.0 v4l2src device=$CAPTURE_DEVICE ! $CAPS !  dtmotion algorithm=cpu-mcd ! videoconvert ! xvimagesink

References

  1. ↑ http://www.ie.tec.ac.cr/palvarado/pmwiki/index.php/Proyectos/DispTEC2?userlang=en
  2. ↑ https://www.cv-foundation.org/openaccess/content_cvpr_workshops_2013/W03/papers/Yi_Detection_of_Moving_2013_CVPR_paper.pdf



For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.


Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.

COMPANY

SUPPORT

ARTIFICIAL INTELLIGENCE

© 2024 RidgeRun. All rights reserved.


Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=GStreamer_DispTEC_Motion_Detection_Plugin&oldid=62787"