GstInference/Benchmarks

GstInference
Introduction
Getting started
Getting the code Building the plugin Quick Starting Guide
Supported architectures
InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3 YoloV2 ResNetV1 MobileNetV2 MobileNetV2 + SSD AlexNet
Supported backends
TensorFlow Tensorflow-Lite Coral from Google Caffe TensorRT ONNXRT ONNXRT ACL ONNXRT OpenVINO
Metadata and Signals
GstClassificationMeta GstInferenceMeta Signals
Overlay Elements
Inference Overlay
Utils Elements
InferenceFilter InferenceCrop InferenceDebug InferenceBin
Legacy pipelines
PC NANO TX2 Xavier IMX8
Example pipelines
Example applications
Classification Detection Hand Pose Recognition
Benchmarks
Model Zoo
Project Status
Releases Roadmap
Contact Us

GstInference Benchmarks

The following benchmarks were run with a source video (1920x1080@60). With the following base GStreamer pipeline, and environment variables:

$ VIDEO_FILE='video.mp4'
$ MODEL_LOCATION='graph_inceptionv1_tensorflow.pb'
$ INPUT_LAYER='input'
$ OUTPUT_LAYER='InceptionV1/Logits/Predictions/Reshape_1'

The environment variables were changed accordingly with the used model (Inception V1,V2,V3 or V4)

$ GST_DEBUG=inception1:1 gst-launch-1.0 filesrc location=$VIDEO_FILE ! decodebin ! videoconvert ! videoscale ! queue ! net.sink_model inceptionv1 name=net model-location=$MODEL_LOCATION backend=tensorflow backend::input-layer=$INPUT_LAYER  backend::output-layer=$OUTPUT_LAYER net.src_model ! perf ! fakesink -v

The Desktop PC had the following specifications:

• Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
• 8 GB RAM
• Cedar [Radeon HD 5000/6000/7350/8350 Series]
• Linux 4.15.0-54-generic x86_64 (Ubuntu 16.04)

The Jetson Xavier power modes used were 2 and 6 (more information: Supported Modes and Power Efficiency )

• View current power mode:

$ sudo /usr/sbin/nvpmodel -q

• Change current power mode:

sudo /usr/sbin/nvpmodel -m x

Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).

Summary

Desktop PC	CPU Library
Model	Framerate	CPU Usage
Inception V1	11.89	48
Inception V2	10.33	65
Inception V3	5.41	90
Inception V4	3.81	94

Jetson Xavier (15W)	CPU Library		GPU Library
Model	Framerate	CPU Usage	Framerate	CPU Usage
Inception V1	8.24	86	52.3	43
Inception V2	6.58	88	39.6	42
Inception V3	2.54	92	17.8	25
Inception V4	1.22	94	9.4	20

Jetson Xavier (30W)	CPU Library		GPU Library
Model	Framerate	CPU Usage	Framerate	CPU Usage
Inception V1	6.41	93	66.27	72
Inception V2	5.11	95	50.59	62
Inception V3	1.96	98	22.95	44
Inception V4	0.98	99	12.14	32

Framerate

thumb

CPU Usage

thumb