GstInference Benchmarks

GstInference
Introduction
Getting started
Getting the code Building the plugin Quick Starting Guide
Supported architectures
InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3 YoloV2 ResNetV1 MobileNetV2 MobileNetV2 + SSD AlexNet
Supported backends
TensorFlow Tensorflow-Lite Coral from Google Caffe TensorRT ONNXRT ONNXRT ACL ONNXRT OpenVINO
Metadata and Signals
GstClassificationMeta GstEmbeddingMeta GstDetectionMeta GstInferenceMeta Signals
Overlay Elements
Inference Overlay
Utils Elements
InferenceFilter InferenceCrop InferenceDebug InferenceBin
Legacy pipelines
PC NANO TX2 Xavier IMX8
Example pipelines
Example applications
Classification Detection Hand Pose Recognition
Benchmarks
Model Zoo
Project Status
Releases Roadmap
Contact Us

GstInference Benchmarks - Introduction

This wiki summarizes a series of benchmarks on different hardware platforms based on the run_benchmark.sh bash script that can be found in the official GstInference repository. The script is based on the following GStreamer pipeline:

#Script to run each model
run_all_models(){

  model_array=(inceptionv1 inceptionv2 inceptionv3 inceptionv4 tinyyolov2 tinyyolov3)
  model_upper_array=(InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3)
  input_array=(input input input input input/Placeholder inputs )
  output_array=(InceptionV1/Logits/Predictions/Reshape_1 Softmax InceptionV3/Predictions/Reshape_1 
  InceptionV4/Logits/Predictions add_8 output_boxes )

  mkdir -p logs/
  rm -f logs/*

  for ((i=0;i<${#model_array[@]};++i)); do
    echo Perf ${model_array[i]}
    gst-launch-1.0 \
    filesrc location=$VIDEO_PATH num-buffers=600 ! decodebin ! videoconvert ! \
    perf print-arm-load=true name=inputperf ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue ! net.sink_bypass \
    ${model_array[i]} backend=$BACKEND name=net backend::input-layer=${input_array[i]} backend::output-layer=${output_array[i]} \
    model-location="${MODELS_PATH}${model_upper_array[i]}_${INTERNAL_PATH}/graph_${model_array[i]}${EXTENSION}" \
    net.src_bypass ! perf print-arm-load=true name=outputperf ! videoconvert ! fakesink sync=false > logs/${model_array[i]}.log
  done
}

Test benchmark video

The following video was used to perform the benchmark tests.
To download the video press right-click on the video and select 'Save video as' and save this on your computer.

Video 1. Test benchmark video

x86

The Desktop PC had the following specifications:

• Intel(R) Core(TM) Core i7-7700HQ CPU @ 2.80GHz
• 12 GB RAM
• Linux 4.15.0-106-generic x86_64 (Ubuntu 16.04)
• GStreamer 1.8.3

Jetson AGX Xavier

The Jetson Xavier power modes used were 2 and 6 (more information: Supported Modes and Power Efficiency)

• View current power mode:

$ sudo /usr/sbin/nvpmodel -q

• Change current power mode:

sudo /usr/sbin/nvpmodel -m x

Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).

Google Coral

The following benchmarks were performed on the Coral Dev Board.