GstInference/Metadatas/GstEmbeddingMeta: Difference between revisions
Maumontero (talk | contribs) No edit summary |
mNo edit summary |
||
(11 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
<noinclude> | <noinclude> | ||
{{GstInference/Head|previous=Metadatas/GstClassificationMeta|next=Metadatas/GstDetectionMeta|keywords=GstInference backends}} | {{GstInference/Head|previous=Metadatas/GstClassificationMeta|next=Metadatas/GstDetectionMeta|keywords=GstInference backends|title=GstInference and GstEmbeddingMeta metadata}} | ||
</noinclude> | </noinclude> | ||
Line 7: | Line 7: | ||
--> | --> | ||
This metadata | This metadata consists of a variable size elements array, that is filed with the embedding produced by the net, like facenet that uses a 128 elements array size. | ||
=Fields= | [[File:gst_inference_GstEmbeddingMetaf.png|500px|center|thumb|GstEmbeddingMeta example]] | ||
==Fields== | |||
The facenet element and embedding overlay uses similar metadata as the classification plugins. GstEmbeddingMeta consist on the following fields: | The facenet element and embedding overlay uses similar metadata as the classification plugins. GstEmbeddingMeta consist on the following fields: | ||
Line 26: | Line 28: | ||
| The embedding produced by the network | | The embedding produced by the network | ||
|} | |} | ||
=Access metadata= | ==Access metadata== | ||
If you want to access this metadata from your custom Gstreamer element instead the process is fairly easy: | If you want to access this metadata from your custom Gstreamer element instead the process is fairly easy: | ||
# Add the [[GstInference/Supported architectures/FaceNet|Facenet]] element to your pipeline | # Add the [[GstInference/Supported architectures/FaceNet|Facenet]] element to your pipeline | ||
Line 36: | Line 36: | ||
FaceNet also raises a signal containing GstClassificationMeta, for details on how to use this signal please check the [[GstInference/Example_Applications/Embedding | example applications]] section. | FaceNet also raises a signal containing GstClassificationMeta, for details on how to use this signal please check the [[GstInference/Example_Applications/Embedding | example applications]] section. | ||
In the following section of code, we add the include like in point 2 and safe the GstClassificationMeta in the class_meta variable like in point 3. | |||
<syntaxhighlight line=1 lang=C> | |||
#include "gst/r2inference/gstinferencemeta.h" | |||
static void | |||
get_buffer(GstPadProbeInfo * info) | |||
{ | |||
GstBuffer *buffer; | |||
GstDetectionMeta *meta; | |||
buffer = gst_pad_probe_info_get_buffer (info); | |||
class_meta = (GstClassificationMeta *) gst_buffer_get_meta (buffer, | |||
GST_DETECTION_META_API_TYPE); | |||
g_print ("Dimension: 0 has embedding %f\n", class_meta->embedding[0]); | |||
} | |||
</syntaxhighlight> | |||
<noinclude> | <noinclude> | ||
{{GstInference/Foot|Metadatas/GstClassificationMeta|Metadatas/GstDetectionMeta}} | {{GstInference/Foot|Metadatas/GstClassificationMeta|Metadatas/GstDetectionMeta}} | ||
</noinclude> | </noinclude> |
Latest revision as of 20:17, 22 October 2020
Make sure you also check GstInference's companion project: R2Inference |
GstInference |
---|
Introduction |
Getting started |
Supported architectures |
InceptionV1 InceptionV3 YoloV2 AlexNet |
Supported backends |
Caffe |
Metadata and Signals |
Overlay Elements |
Utils Elements |
Legacy pipelines |
Example pipelines |
Example applications |
Benchmarks |
Model Zoo |
Project Status |
Contact Us |
|
This metadata consists of a variable size elements array, that is filed with the embedding produced by the net, like facenet that uses a 128 elements array size.
Fields
The facenet element and embedding overlay uses similar metadata as the classification plugins. GstEmbeddingMeta consist on the following fields:
field | type | description |
---|---|---|
num_dimensions | gint | The number of labels outputted by the model. This can vary from model to model. Facenet uses 128. |
embedding | gdouble * | The embedding produced by the network |
Access metadata
If you want to access this metadata from your custom Gstreamer element instead the process is fairly easy:
- Add the Facenet element to your pipeline
- Include GstInference metadata header:
#include "gst/r2inference/gstinferencemeta.h"
- Get a GstClassificationMeta object from the buffer:
class_meta = (GstClassificationMeta *) gst_buffer_get_meta (frame->buffer, GST_CLASSIFICATION_META_API_TYPE);
FaceNet also raises a signal containing GstClassificationMeta, for details on how to use this signal please check the example applications section.
In the following section of code, we add the include like in point 2 and safe the GstClassificationMeta in the class_meta variable like in point 3.
#include "gst/r2inference/gstinferencemeta.h" static void get_buffer(GstPadProbeInfo * info) { GstBuffer *buffer; GstDetectionMeta *meta; buffer = gst_pad_probe_info_get_buffer (info); class_meta = (GstClassificationMeta *) gst_buffer_get_meta (buffer, GST_DETECTION_META_API_TYPE); g_print ("Dimension: 0 has embedding %f\n", class_meta->embedding[0]); }