GstInference/Benchmarks: Difference between revisions

From RidgeRun Developer Wiki
Line 162: Line 162:
     cursor: pointer;
     cursor: pointer;
   }
   }
  </style>
</style>
    
   <div id="Buttons_Model" style="margin: auto; width: 1200px; height: auto;">
   <button class="button" id="show_inceptionv1">Show InceptionV1 </button>
   <button class="button" id="show_inceptionv1">Show InceptionV1 </button>
   <button class="button" id="show_inceptionv2">Show InceptionV2 </button>
   <button class="button" id="show_inceptionv2">Show InceptionV2 </button>
   <button class="button" id="show_inceptionv3">Show InceptionV3 </button>
   <button class="button" id="show_inceptionv3">Show InceptionV3 </button>
   <button class="button" id="show_inceptionv4">Show InceptionV4 </button>
   <button class="button" id="show_inceptionv4">Show InceptionV4 </button>
 
  <button class="button" id="show_tinyyolov2">Show TinyYoloV2 </button>
  <button class="button" id="show_tinyyolov3">Show TinyYoloV3 </button>
</div>
<br><br>
<br><br>
<div id="chart_div" style="width: 800px; height: 500px;"></div>
<div id="chart_div" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<br><br>
<div id="Buttons_Backend" style="margin: auto; width: 420px; height: auto;">
<button class="button" id="show_tensorflow">Show Tensorflow </button>
<button class="button" id="show_tensorflow">Show Tensorflow </button>
<button class="button" id="show_tflite">Show Tensorflow-Lite </button>
<button class="button" id="show_tflite">Show Tensorflow-Lite </button>
<div id="chart_div1" style="width: 800px; height: 500px;"></div>
</div>
<div id="chart_div1" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<br><br>
<div id="Buttons_Platform" style="margin: auto; width: 1050px; height: auto;">
<button class="button" id="show_x86">Show x86 </button>
<button class="button" id="show_x86">Show x86 </button>
<button class="button" id="show_x86_gpu">Show X86+GPU </button>
<button class="button" id="show_x86_gpu">Show X86+GPU </button>
Line 182: Line 187:
<button class="button" id="show_coral">Show Coral </button>
<button class="button" id="show_coral">Show Coral </button>
<button class="button" id="show_coral_tpu">Show Coral+TPU </button>
<button class="button" id="show_coral_tpu">Show Coral+TPU </button>
<div id="chart_div2" style="width: 800px; height: 500px;"></div>
</div>
<br><br>
<div id="chart_div2" style="margin: auto; width: 800px; height: 500px;"></div>
 
<script>
<script>
       google.charts.load('current', {'packages':['corechart', 'bar']});
       google.charts.load('current', {'packages':['corechart', 'bar']});
Line 205: Line 211:
           'InceptionV3 \n Tensorflow-Lite',
           'InceptionV3 \n Tensorflow-Lite',
           'InceptionV4 \n Tensorflow',
           'InceptionV4 \n Tensorflow',
           'InceptionV4 \n Tensorflow-Lite'],
           'InceptionV4 \n Tensorflow-Lite',
           ['x86',      20.164, 19.545, 18.164, 17.642, 16.164, 15.642, 14.164, 13.642],
          'TinyYoloV2 \n Tensorflow',
           ['x86+GPU',  25.821, 24.587, 23.164, 22.642, 21.164, 20.642, 19.164, 18.642],
          'TinyYoloV2 \n Tensorflow-Lite',
           ['TX2',      24.821, 23.587, 22.164, 21.642, 20.164, 19.642, 18.164, 17.642],
          'TinyYoloV3 \n Tensorflow',
           ['TX2-GPU',  27.821, 26.587, 25.164, 24.642, 23.164, 22.642, 21.164, 20.642],
          'TinyYoloV3 \n Tensorflow-Lite'],
           ['Coral',    16.111, 15.966, 14.164, 13.642, 12.164, 11.642, 10.164, 9.642],
           ['x86',      20.164, 19.545, 18.164, 17.642, 16.164, 15.642, 14.164, 13.642, 28.164, 16.642, 16.164, 15.642],
           ['Coral+TPU', 18.111, 17.966, 16.164, 15.642, 14.164, 13.642, 12.164, 11.642]
           ['x86+GPU',  25.821, 24.587, 23.164, 22.642, 21.164, 20.642, 19.164, 18.642, 25.164, 15.642, 16.164, 15.642],
           ['TX2',      24.821, 23.587, 22.164, 21.642, 20.164, 19.642, 18.164, 17.642, 26.164, 14.642, 16.164, 15.642],
           ['TX2-GPU',  27.821, 26.587, 25.164, 24.642, 23.164, 22.642, 21.164, 20.642, 25.164, 13.642, 16.164, 15.642],
           ['Coral',    16.111, 15.966, 14.164, 13.642, 12.164, 11.642, 10.164, 9.642, 36.164, 12.642, 16.164, 15.642],
           ['Coral+TPU', 18.111, 17.966, 16.164, 15.642, 14.164, 13.642, 12.164, 11.642, 31.164, 11.642, 16.164, 15.642]
         ]);
         ]);
         var table_model_platform = google.visualization.arrayToDataTable([
         var table_model_platform = google.visualization.arrayToDataTable([
Line 227: Line 237:
           'Tensorflow-Lite \n Coral',
           'Tensorflow-Lite \n Coral',
           'Tensorflow-Lite \n Coral+TPU'],
           'Tensorflow-Lite \n Coral+TPU'],
           ['InceptionV1',   20.164, 25.642, 24.164, 27.642, 16.164, 18.642, 19.164, 24.642, 23.164, 26.642, 15.164, 17.642],
           ['InceptionV1', 20.164, 25.642, 24.164, 27.642, 16.164, 18.642, 19.164, 24.642, 23.164, 26.642, 15.164, 17.642],
           ['InceptionV2',   18.821, 23.587, 22.164, 25.642, 14.164, 16.642, 17.164, 22.642, 21.164, 24.642, 13.164, 15.642],
           ['InceptionV2', 18.821, 23.587, 22.164, 25.642, 14.164, 16.642, 17.164, 22.642, 21.164, 24.642, 13.164, 15.642],
           ['InceptionV3',   16.086, 21.286, 20.164, 23.642, 12.164, 14.642, 15.164, 20.642, 19.164, 22.642, 11.164, 13.642],
           ['InceptionV3', 16.086, 21.286, 20.164, 23.642, 12.164, 14.642, 15.164, 20.642, 19.164, 22.642, 11.164, 13.642],
           ['InceptionV4',   14.977,  19.153, 18.164, 21.642, 10.164, 12.642, 13.164, 18.642, 17.164, 20.642, 9.164, 11.642]
           ['InceptionV4', 14.977, 19.153, 18.164, 21.642, 10.164, 12.642, 13.164, 18.642, 17.164, 20.642, 9.164, 11.642],
          ['TinyYoloV2',  15.977, 16.153, 16.164, 23.642, 12.164, 13.642, 13.164, 17.642, 17.164, 28.642, 19.164, 15.642],
          ['TinyYoloV3',  16.977, 19.153, 18.164, 21.642, 10.164, 12.642, 13.164, 18.642, 17.164, 20.642, 9.164, 11.642]
         ]);
         ]);
         var table_model_backend = google.visualization.arrayToDataTable([
         var table_model_backend = google.visualization.arrayToDataTable([
Line 246: Line 258:
           'Tensorflow-Lite \n Coral',
           'Tensorflow-Lite \n Coral',
           'Tensorflow-Lite \n Coral+TPU'],
           'Tensorflow-Lite \n Coral+TPU'],
           ['InceptionV1',   20.164, 25.642, 24.164, 27.642, 16.164, 18.642, 19.164, 24.642, 23.164, 26.642, 15.164, 17.642],
           ['InceptionV1', 20.164, 25.642, 24.164, 27.642, 16.164, 18.642, 19.164, 24.642, 23.164, 26.642, 15.164, 17.642],
           ['InceptionV2',   18.821, 23.587, 22.164, 25.642, 14.164, 16.642, 17.164, 22.642, 21.164, 24.642, 13.164, 15.642],
           ['InceptionV2', 18.821, 23.587, 22.164, 25.642, 14.164, 16.642, 17.164, 22.642, 21.164, 24.642, 13.164, 15.642],
           ['InceptionV3',   16.086,  21.286, 20.164, 23.642, 12.164, 14.642, 15.164, 20.642, 19.164, 22.642, 11.164, 13.642],
           ['InceptionV3', 16.086,  21.286, 20.164, 23.642, 12.164, 14.642, 15.164, 20.642, 19.164, 22.642, 11.164, 13.642],
           ['InceptionV4',   14.977,  19.153, 18.164, 21.642, 10.164, 12.642, 13.164, 18.642, 17.164, 20.642, 9.164, 11.642]
           ['InceptionV4', 14.977,  19.153, 18.164, 21.642, 10.164, 12.642, 13.164, 18.642, 17.164, 20.642, 9.164, 11.642],
          ['TinyYoloV2',  15.977, 16.153, 16.164, 23.642, 12.164, 13.642, 13.164, 17.642, 17.164, 28.642, 19.164, 15.642],
          ['TinyYoloV3',  16.977, 19.153, 18.164, 21.642, 10.164, 12.642, 13.164, 18.642, 17.164, 20.642, 9.164, 11.642]
         ]);
         ]);


Line 313: Line 327:
         function init_charts(){
         function init_charts(){
           view.setColumns([0,1,2]);
           view.setColumns([0,1,2]);
           view.hideColumns([3,4,5,6,7,8]);
           view.hideColumns([3,4,5,6,7,8,9,10,11,12]);
           materialChart.draw(view, materialOptions);
           materialChart.draw(view, materialOptions);
           view1.setColumns([0,1,2,3,4,5,6]);
           view1.setColumns([0,1,2,3,4,5,6]);
Line 329: Line 343:
         show_inceptionv1.onclick = function () {
         show_inceptionv1.onclick = function () {
           view.setColumns([0,1,2]);
           view.setColumns([0,1,2]);
           view.hideColumns([3,4,5,6,7,8]);
           view.hideColumns([3,4,5,6,7,8,9,10,11,12]);
           materialChart.draw(view, materialOptions);
           materialChart.draw(view, materialOptions);
         }
         }
Line 335: Line 349:
         show_inceptionv2.onclick = function() {
         show_inceptionv2.onclick = function() {
           view.setColumns([0,3,4]);
           view.setColumns([0,3,4]);
           view.hideColumns([1,2,5,6,7,8]);
           view.hideColumns([1,2,5,6,7,8,9,10,11,12]);
           materialChart.draw(view, materialOptions);
           materialChart.draw(view, materialOptions);
         }
         }
Line 341: Line 355:
         show_inceptionv3.onclick = function() {
         show_inceptionv3.onclick = function() {
           view.setColumns([0,5,6]);
           view.setColumns([0,5,6]);
           view.hideColumns([1,2,3,4,7,8]);
           view.hideColumns([1,2,3,4,7,8,9,10,11,12]);
           materialChart.draw(view, materialOptions);
           materialChart.draw(view, materialOptions);
         }
         }
Line 347: Line 361:
         show_inceptionv4.onclick = function() {
         show_inceptionv4.onclick = function() {
           view.setColumns([0,7,8]);
           view.setColumns([0,7,8]);
           view.hideColumns([1,2,3,4,5,6]);
           view.hideColumns([1,2,3,4,5,6,9,10,11,12]);
          materialChart.draw(view, materialOptions);
        }
        var show_tinyyolov2 = document.getElementById("show_tinyyolov2");
        show_tinyyolov2.onclick = function() {
          view.setColumns([0,9,10]);
          view.hideColumns([1,2,3,4,5,6,7,8,11,12]);
          materialChart.draw(view, materialOptions);
        }
        var show_tinyyolov3 = document.getElementById("show_tinyyolov3");
        show_tinyyolov3.onclick = function() {
          view.setColumns([0,11,12]);
          view.hideColumns([1,2,3,4,5,6,7,8,9,10]);
           materialChart.draw(view, materialOptions);
           materialChart.draw(view, materialOptions);
         }
         }

Revision as of 14:44, 27 March 2020




Previous: Example Applications/DispTec Index Next: Model Zoo


GstInference
GstInference
Introduction
Getting started
Supported architectures

InceptionV1

InceptionV3

YoloV2

AlexNet
Supported backends

Caffe

Metadata and Signals
Overlay Elements
Utils Elements
Legacy pipelines
Example pipelines
Example applications
Benchmarks
Model Zoo
Project Status
Contact Us

Coral from Google



GstInference Benchmarks

The following benchmarks were run with a source video (1920x1080@60). With the following base GStreamer pipeline, and environment variables: 

$ VIDEO_FILE='video.mp4'
$ MODEL_LOCATION='graph_inceptionv1_tensorflow.pb'
$ INPUT_LAYER='input'
$ OUTPUT_LAYER='InceptionV1/Logits/Predictions/Reshape_1'

The environment variables were changed accordingly with the used model (Inception V1,V2,V3 or V4) 

GST_DEBUG=inception1:1 gst-launch-1.0 filesrc location=$VIDEO_FILE ! decodebin ! videoconvert ! videoscale ! queue ! net.sink_model inceptionv1 name=net model-location=$MODEL_LOCATION backend=tensorflow backend::input-layer=$INPUT_LAYER  backend::output-layer=$OUTPUT_LAYER net.src_model ! perf ! fakesink -v

The Desktop PC had the following specifications:

  • Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
  • 8 GB RAM
  • Cedar [Radeon HD 5000/6000/7350/8350 Series]
  • Linux 4.15.0-54-generic x86_64 (Ubuntu 16.04)

The Jetson Xavier power modes used were 2 and 6 (more information: Supported Modes and Power Efficiency)

  • View current power mode:
$ sudo /usr/sbin/nvpmodel -q
  • Change current power mode:
sudo /usr/sbin/nvpmodel -m x

Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).

Summary

Desktop PC CPU Library
Model Framerate CPU Usage
Inception V1 11.89 48
Inception V2 10.33 65
Inception V3 5.41 90
Inception V4 3.81 94
Jetson Xavier (15W) CPU Library GPU Library
Model Framerate CPU Usage Framerate CPU Usage
Inception V1 8.24 86 52.3 43
Inception V2 6.58 88 39.6 42
Inception V3 2.54 92 17.8 25
Inception V4 1.22 94 9.4 20
Jetson Xavier (30W) CPU Library GPU Library
Model Framerate CPU Usage Framerate CPU Usage
Inception V1 6.41 93 66.27 72
Inception V2 5.11 95 50.59 62
Inception V3 1.96 98 22.95 44
Inception V4 0.98 99 12.14 32

Framerate

thumb
thumb

CPU Usage

thumb
thumb

TensorFlow Lite Benchmarks







Test benchmark video

The following video was used to perform the benchmark tests.
To download the video press right click on the video and select 'Save video as' and save this in your computer.

Test benchmark video


Previous: Example Applications/DispTec Index Next: Model Zoo



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=GstInference/Benchmarks&oldid=28575"