GstInference and Coral backend

GstInference
Introduction
Getting started
Getting the code Building the plugin Quick Starting Guide
Supported architectures
InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3 YoloV2 ResNetV1 MobileNetV2 MobileNetV2 + SSD AlexNet
Supported backends
TensorFlow Tensorflow-Lite Coral from Google Caffe TensorRT ONNXRT ONNXRT ACL ONNXRT OpenVINO
Metadata and Signals
GstClassificationMeta GstEmbeddingMeta GstDetectionMeta GstInferenceMeta Signals
Overlay Elements
Inference Overlay
Utils Elements
InferenceFilter InferenceCrop InferenceDebug InferenceBin
Legacy pipelines
PC NANO TX2 Xavier IMX8
Example pipelines
Example applications
Classification Detection Hand Pose Recognition
Benchmarks
Model Zoo
Project Status
Releases Roadmap
Contact Us

The Edge TPU is an ASIC designed by Google to provide high-performance machine learning applications that are running on embedded devices. The EdgeTPU library extends the TensorFlow Lite framework. For more information about the Edge TPU hardware go to the Coral dev board page.

To use the Edge TPU backend on Gst-Inference be sure to run the R2Inference configure with the flag -Denable-coral=true. Then, use the property backend=coral on the Gst-Inference plugins. GstInference depends on the C++ API of Tensorflow-Lite.

Installation

GstInference depends on the C++ API of Tensorflow-Lite. For installation steps, follow the steps in R2Inference/Building the library section.

TensorFlow Python API and utilities can be installed with python pip, but it is not needed by GstInference.

Enabling the backend

To enable Tensorflow-Lite as a backend for GstInference you need to install R2Inference with TensorFlow-Lite, which is a dependency, and EdgeTPU support. To do this, use the option -Denable-coral=true while following this wiki

Properties

TensorFlow Lite API Reference has full documentation of the Tensorflow-Lite C++ API. Gst-Inference uses only the C++ API of Tensorflow-Lite and R2Inference takes care of devices and loading the models.

The following syntax is used to change backend options on Gst-Inference plugins:

backend::<property>

For example to change the backend to use Tensorflow-Lite you need to run the pipeline like this:

gst-launch-1.0 \ 
mobilenetv2 name=net model-location=mobilenet_v2_1.0_224_quant_edgetpu.tflite backend=coral \
filesrc location=video_stream.mp4 ! decodebin ! videoconvert ! videoscale ! queue ! tee name=t \
t. ! queue ! videoconvert ! videoscale !  net.sink_model \
t. ! queue ! videoconvert ! net.sink_bypass \
 net.src_model ! fakesink

To learn more about the EdgeTPU C++ API, please check the Tensorflow-Lite API section on the R2Inference subwiki.

Tools

The TensorFlow Python API installation includes a tool named Tensorboard, that can be used to visualize a model. If you want some examples and a more complete description please check the Tools section on the R2Inference wiki.