GstInference with GstClassificationMeta metadata

GstInference
Introduction
Getting started
Getting the code Building the plugin Quick Starting Guide
Supported architectures
InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3 YoloV2 ResNetV1 MobileNetV2 MobileNetV2 + SSD AlexNet
Supported backends
TensorFlow Tensorflow-Lite Coral from Google Caffe TensorRT ONNXRT ONNXRT ACL ONNXRT OpenVINO
Metadata and Signals
GstClassificationMeta GstEmbeddingMeta GstDetectionMeta GstInferenceMeta Signals
Overlay Elements
Inference Overlay
Utils Elements
InferenceFilter InferenceCrop InferenceDebug InferenceBin
Legacy pipelines
PC NANO TX2 Xavier IMX8
Example pipelines
Example applications
Classification Detection Hand Pose Recognition
Benchmarks
Model Zoo
Project Status
Releases Roadmap
Contact Us

This metadata consists of a probability for each of the model's trained classes.

GstClassificationMeta example

Fields

All classification elements on GstInference use the same metadata standard. GstClassificationMeta consist on the following fields:

field	type	description
num_labels	gint	The number of labels outputted by the model. This can vary from model to model.
label_probs	gdouble *	The probability for each label.

Access metadata

If you want to access this metadata from your custom Gstreamer element instead the process is fairly easy:

1. Add a GstInference classification element such as InceptionV4 to your pipeline
2. Include GstInference metadata header: #include "gst/r2inference/gstinferencemeta.h"
3. Get a GstClassificationMeta object from the buffer: class_meta = (GstClassificationMeta *) gst_buffer_get_meta (frame->buffer, GST_CLASSIFICATION_META_API_TYPE);

All GstInference classification elements also raise a signal containing GstClassificationMeta, for details on how to use this signal please check the example applications section.

In the following section of code, we add the include like in point 2 and safe the GstClassificationMeta in the class_meta variable like in point 3.

#include "gst/r2inference/gstinferencemeta.h"
static void 
get_buffer(GstPadProbeInfo * info)
{
  GstBuffer *buffer;
  GstDetectionMeta *meta;
  buffer = gst_pad_probe_info_get_buffer (info);
  class_meta = (GstClassificationMeta *) gst_buffer_get_meta (buffer,
                  GST_DETECTION_META_API_TYPE);

  g_print ("Class: 0 has probability %f\n", class_meta->label_probs[0]);
}