GstInference Detection application

Previous: Example Applications/Classification

This example receives an input video file and detects objects in each buffer, the possible objects to detect are available in the classes section. For each frame the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with every detected object marked with a label and a square. Note that the image currently being displayed not necessarily matches the one being handled by the signal. The display is meant for visualization and debugging purposes.

The detection example is based on TinyYoloV2, trained using the Pascal VOC dataset, more information can be found in YOLO: Real-Time Object Detection. A pre-trained model can be downloaded from the GstInference Model Zoo

This examples serves both as an example and as a starting point for detection application.

Pascal VOC Classes

The VOC dataset used for training TinyYoloV2 supports 20 classes for inference. Classes according to label ID:

aeroplane
bicycle
bird
boat
bottle
bus
car
cat
chair
cow
diningtable
dog
horse
motorbike
person
pottedplant
sheep
sofa
train
tvmonitor

Building the Example

The example builds along with the GstInference project. Make sure you follow the instructions in Building the Plug-In to make sure all the dependencies are correctly fulfilled.

Once the project is built the example may be built independently by running make within the example directory.

cd tests/examples/detection
make

The example is not meant to be installed.

Running the Example

The detection application provides a series of cmdline options to control the behavior of the example. The basic usage is:

./detection -m MODEL -f FILE -b BACKEND [-v]

-m|--model: Mandatory. Path to the TinyYoloV2 trained model
-f|--file: Mandatory. Path to the video file to be used.; If it is not set, a camera stream will be used.
-b|--backend: Mandatory. Name of the backed to be used. See Supported Backends for a list of possible options.
-v: Optional. Run verbosely.

You may always run --help for further details.

./detection --help
Usage:
  detection [OPTIONS] - GstInference Detection Example

Help Options:
  -h, --help                        Show help options
  --help-all                        Show all help options
  --help-gst                        Show GStreamer Options

Application Options:
  -v, --verbose                     Be verbose
  -m, --model                       Model path
  -f, --file                        File path
  -b, --backend                     Backend used for inference, example: tensorflow

Extending the Application

The example is composed of the following main files:

gstdetection.c: Source file with GStreamer logic
customlogic.c: Placeholder for custom logic
customlogic.h: Header file for custom logic source
Makefile: Build script
detection: Executable generated on build

The application can be extended by filling in the placeholders in customlogic.c. In the most simple cases, the information provided in handle_predictionshould suffice to react to the prediction.

void
handle_prediction (unsigned char *image, 
                   int width, 
                   int height,
                   unsigned int size,
                   BoundingBox * boxes,
                   int num_boxes)
{
    /* FILLME: Handle image and prediction here */
}

image: Image data in RGB raster format.
width: The width of the image, in pixels
height: The height of the image, in pixels
size: The size of the total image, in bytes
boxes: An array of boxes with all detected objects in the frame
num_boxes: The length of the boxes array

Troubleshooting

The first debug level is GStreamer debug. You may execute the application with debug enabled by running:

./detection --gst-debug=2

For advanced support, please Contact RidgeRun OR email to support@ridgerun.com with the output of the following command:

./detection --gst-debug=2,videoinference:6,tinyyolo:6

Reporting a Bug

Please feel free to report bugs using GitHub's issue tracker.

Previous: Example Applications/Classification

Index

Next: Example Applications/DispTec

❯