GstInference inferencebin
 | Make sure you also check GstInference's companion project: R2Inference |
| GstInference |
|---|
 |
| Introduction |
| Getting started |
| Supported architectures |
|
InceptionV1 InceptionV3 YoloV2 AlexNet |
| Supported backends |
|
Caffe |
| Metadata and Signals |
| Overlay Elements |
| Utils Elements |
| Legacy pipelines |
| Example pipelines |
| Example applications |
| Benchmarks |
| Model Zoo |
| Project Status |
| Contact Us |
 |
GstInference inferencebin description
InferenceBin is GStreamer bin that bundles several linked elements internally. Think of how the playbin element works, this element in particular allows to playback media contained in many formats in a generic fashion. To do this playbin links internally the source element (network streaming, video file), the decoder/demuxer and or audio/video sinks.
Similarly InferenceBin can be used to grouped together multiples GstInference elements (inceptionv2, tinyyolov3, inferencefilter, inferenceoverlay, etc...) into one logical element inside the pipeline. The following figure illustrates the internal structure of inferencebin:
Why use inferencebin?
The inferencebin element was created mainly for reducing pipeline descriptions and boilerplate code inside applications. So it is expected that you will see it facilitates using the GstInference project, specially when you are just getting familiar with the different elements the plugin provides.
Advantages
- Provides an "all-in-one" element for quick pipeline construction
- Easy control of internal behavior through element properties
Disavantages
- You may lose flexibility and fine tuning control because inferencebin assumes a pipeline structure
Inferencebin Properties
How the elements are linked together inside the InferenBin depends of the configuration of the its properties.
The following table summarizes the properties of the element:
| Property | Value | Description |
|---|---|---|
| arch | String tinyyolov2: empty |
The network architecture. For example: tinyyolov2 |
| backend | String. tensorflow: empty |
The inference backend. For example: tensorflow |
| model-location | String. Default: null |
Path to model file. |
| input-layer | String. Default: null |
The name of the input of the mode. |
| output-layer | String. Default: null |
The name of the output of the mode. |
| labels | String. Default: null |
The labels used to train the model |
| crop | Boolean. Default: false |
Whether or not to crop out objects in the current prediction. |
| overlay | Boolean. Default: true |
Whether or not to overlay predictions on the buffers |
| filter | Integer. Default: -1 |
The filter to apply to the inference. |
| scaler | String. Default: videoscale |
Bin description to use as video scaler. |
| converter | String. Default: videoconvert |
Bin description to use as color space converter. |
Example
Simple pipeline
# To extract pipeline diagram export GST_DEBUG_DUMP_DOT_DIR=/tmp/ #Inference bin parameters ARCH='tinyyolov2' BACKEND='tensorflow' MODEL_LOCATION='graph_tinyyolov2_tensorflow.pb' INPUT='input/Placeholder' OUTPUT='add_8' LABELS='labels.txt' CROP=false OVERLAY=true FILTER=-1 gst-launch-1.0 v4l2src ! inferencebin arch=$ARCH model-location=$MODEL_LOCATION backend=$BACKEND input-layer=$INPUT output-layer=$OUTPUT \ labels=$LABELS crop=$CROP overlay=$OVERLAY filter=$FILTER ! videoconvert ! ximagesink sync=false
After extracting the dot file from the pipeline, it can be seen it is equivalent to:
gst-launch-1.0 v4l2src ! inferencefilter ! inferencedebug ! videoconvert qos=true ! tee name=t \ t. ! queue max-size-buffers=3 ! arch.sink_bypass \ t. ! queue ! inferencecrop enable=false ! videoscale qos=true ! arch.sink_model \ tinyyolov2 name=arch model-location=$MODEL_LOCATION labels=$LABELS backend::input-layer=$INPUT backend::output-layer=$OUTPUT \ ! queue ! inferencedebug ! inferenceoverlay qos=true ! videoconvert qos=true ! xvimagesink sync=false max-lateness=20000000 qos=true
The following diagram illustrates the pipeline above.
Advanced pipeline
The inferencebin element can also be used to simplify greatly a pipeline that implies a concatenation of two models. In this example the pipeline performs a classification (inceptionv1) over a previous detection (tinyyolov2). Notice that the first inferencebin bin is used just to perform the detection while the second inferencebin is used to filter the inference metadata, crop the image and draw the overlay.
#Properties for detection ARCHDET='tinyyolov2' BACKENDDET='tensorflow' MODELLOCATIONDET='graph_tinyyolov2_tensorflow.pb' INPUTDET="input/Placeholder" OUTPUTDET="add_8" LABELSDET="labels.txt" CROPDET=false OVERLAYDET=false FILTERDET=-1 #Properties for classification ARCHCLASS='inceptionv1' BACKENDCLASS='tensorflow' MODELLOCATIONCLASS='graph_inceptionv1_tensorflow.pb' INPUTCLASS='input' OUTPUTCLASS='InceptionV1/Logits/Predictions/Reshape_1' LABELSCLASS="imagenet_labels.txt" CROPCLASS=true OVERLAYCLASS=true FILTERCLASS=14 GST_DEBUG=2,inferencebin:6 gst-launch-1.0 multifilesrc location=person.jpg start-index=0 stop-index=0 loop=true ! jpegparse ! jpegdec ! videoconvert ! queue ! \ inferencebin arch=$ARCHDET model-location=$MODELLOCATIONDET backend=$BACKENDDET input-layer=$INPUTDET output-layer=$OUTPUTDET labels=$LABELSDET crop=$CROPDET \ overlay=$OVERLAYDET filter=$FILTERDET ! queue ! inferencebin arch=$ARCHCLASS model-location=$MODELLOCATIONCLASS backend=$BACKENDCLASS input-layer=$INPUTCLASS \ output-layer=$OUTPUTCLASS labels=$LABELSCLASS crop=$CROPCLASS overlay=$OVERLAYCLASS filter=$FILTERCLASS ! videoconvert ! xvimagesink sync=false
The following diagram illustrates the pipeline above. If you want to know about the internal structure of inferencebin check the description section of this wiki page.
