GstInference/Benchmarks: Difference between revisions

From RidgeRun Developer Wiki
No edit summary
No edit summary
Tag: Manual revert
 
(40 intermediate revisions by 4 users not shown)
Line 1: Line 1:
<noinclude>
<noinclude>
{{GstInference/Head|previous=Example Applications/DispTec|next=Model Zoo|keywords=GstInference gstreamer pipelines, Inference gstreamer pipelines, NCSDK Inference gstreamer pipelines, GoogLeNet, TinyYolo v2, GoogLeNet x2, TensorFlow backend|title=GstInference Benchmarks}}
{{GstInference/Head|previous=Example Applications/DispTec|next=Model Zoo|metakeywords=GstInference gstreamer pipelines, Inference gstreamer pipelines, NCSDK Inference gstreamer pipelines, GoogLeNet, TinyYolo v2, GoogLeNet x2, TensorFlow backend|title=GstInference Benchmarks}}
</noinclude>
</noinclude>


Line 11: Line 11:
</html>
</html>


== GstInference Benchmarks ==
== GstInference Benchmarks - Introduction ==
The following benchmarks were run with a source video (1920x1080@60). With the following base [https://www.ridgerun.com/gstreamer GStreamer] pipeline, and environment variables:


<source lang="bash">
This wiki summarizes a series of benchmarks on different hardware platforms based on the [https://github.com/RidgeRun/gst-inference/blob/master/tests/benchmark/run_benchmark.sh run_benchmark.sh] bash script that can be found in the official [https://github.com/RidgeRun/gst-inference GstInference repository]. The script is based on the following GStreamer pipeline:
$ VIDEO_FILE='video.mp4'
$ MODEL_LOCATION='graph_inceptionv1_tensorflow.pb'
$ INPUT_LAYER='input'
$ OUTPUT_LAYER='InceptionV1/Logits/Predictions/Reshape_1'
</source>
The environment variables were changed accordingly with the used model (Inception V1,V2,V3 or V4)
 
<source lang="bash">
GST_DEBUG=inception1:1 gst-launch-1.0 filesrc location=$VIDEO_FILE ! decodebin ! videoconvert ! videoscale ! queue ! net.sink_model inceptionv1 name=net model-location=$MODEL_LOCATION backend=tensorflow backend::input-layer=$INPUT_LAYER  backend::output-layer=$OUTPUT_LAYER net.src_model ! perf ! fakesink -v
</source>
 
The Desktop PC had the following specifications:
*Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
*8 GB RAM
*Cedar [Radeon HD 5000/6000/7350/8350 Series]
*Linux 4.15.0-54-generic x86_64 (Ubuntu 16.04)
 
The Jetson Xavier power modes used were 2 and 6 (more information: [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fpower_management_jetson_xavier.html%23wwpID0E0OM0HA Supported Modes and Power Efficiency])
 
*View current power mode:
 
<source lang="bash">
$ sudo /usr/sbin/nvpmodel -q
</source>
 
*Change current power mode:


<source lang="bash">
<source lang="bash">
sudo /usr/sbin/nvpmodel -m x
#Script to run each model
</source>
run_all_models(){
Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).


=== Summary ===  
  model_array=(inceptionv1 inceptionv2 inceptionv3 inceptionv4 tinyyolov2 tinyyolov3)
  model_upper_array=(InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3)
  input_array=(input input input input input/Placeholder inputs )
  output_array=(InceptionV1/Logits/Predictions/Reshape_1 Softmax InceptionV3/Predictions/Reshape_1
  InceptionV4/Logits/Predictions add_8 output_boxes )


{| class="wikitable" style="display: inline-table;"
  mkdir -p logs/
! style="font-weight:bold; background-color:#efefef; color:#000000;" | Desktop PC
  rm -f logs/*
! colspan="2" style="text-align: center; font-weight:bold; background-color:#efefef; color:#000000;" | CPU Library
|-
| style="background-color:#e98d44; color:#000000;" | Model
| style="background-color:#e98d44; color:#000000;" | Framerate
| style="background-color:#e98d44; color:#000000;" | CPU Usage
|-
| style="background-color:#e98d44; color:#000000;" | Inception V1
| style="background-color:#fee3cd; color:#000000;" | 11.89
| style="background-color:#fee3cd; color:#000000;" | 48
|-
| style="background-color:#e98d44; color:#000000;" | Inception V2
| style="background-color:#fee3cd; color:#000000;" | 10.33
| style="background-color:#fee3cd; color:#000000;" | 65
|-
| style="background-color:#e98d44; color:#000000;" | Inception V3
| style="background-color:#fee3cd; color:#000000;" | 5.41
| style="background-color:#fee3cd; color:#000000;" | 90
|-
| style="background-color:#e98d44; color:#000000;" | Inception V4
| style="background-color:#fee3cd; color:#000000;" | 3.81
| style="background-color:#fee3cd; color:#000000;" | 94
|}


{| class="wikitable" style="display: inline-table;"
  for ((i=0;i<${#model_array[@]};++i)); do
! style="font-weight:bold; background-color:#efefef; color:#000000;" | Jetson Xavier (15W)
    echo Perf ${model_array[i]}
! colspan="2" style="text-align: center; font-weight:bold; background-color:#efefef; color:#000000;" | CPU Library
     gst-launch-1.0 \
! colspan="2" style="text-align: center; font-weight:bold; background-color:#efefef; color:#000000;" | GPU Library
     filesrc location=$VIDEO_PATH num-buffers=600 ! decodebin ! videoconvert ! \
|-
     perf print-arm-load=true name=inputperf ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue ! net.sink_bypass \
| style="background-color:#2c79d3; color:#000000;" | Model
    ${model_array[i]} backend=$BACKEND name=net backend::input-layer=${input_array[i]} backend::output-layer=${output_array[i]} \
| style="background-color:#2c79d3; color:#000000;" | Framerate
     model-location="${MODELS_PATH}${model_upper_array[i]}_${INTERNAL_PATH}/graph_${model_array[i]}${EXTENSION}" \
| style="background-color:#2c79d3; color:#000000;" | CPU Usage
    net.src_bypass ! perf print-arm-load=true name=outputperf ! videoconvert ! fakesink sync=false > logs/${model_array[i]}.log
| style="background-color:#2c79d3; color:#000000;" | Framerate
  done
| style="background-color:#2c79d3; color:#000000;" | CPU Usage
}
|-
</source>  
| style="background-color:#2c79d3; color:#000000;" | Inception V1
| style="background-color:#c5daf6; color:#000000;" | 8.24
| style="background-color:#c5daf6; color:#000000;" | 86
| style="background-color:#c5daf6; color:#000000;" | 52.3
| style="background-color:#c5daf6; color:#000000;" | 43
|-
| style="background-color:#2c79d3; color:#000000;" | Inception V2
| style="background-color:#c5daf6; color:#000000;" | 6.58
| style="background-color:#c5daf6; color:#000000;" | 88
| style="background-color:#c5daf6; color:#000000;" | 39.6
| style="background-color:#c5daf6; color:#000000;" | 42
|-
| style="background-color:#2c79d3; color:#000000;" | Inception V3
| style="background-color:#c5daf6; color:#000000;" | 2.54
| style="background-color:#c5daf6; color:#000000;" | 92
| style="background-color:#c5daf6; color:#000000;" | 17.8
| style="background-color:#c5daf6; color:#000000;" | 25
|-
| style="background-color:#2c79d3; color:#000000;" | Inception V4
| style="background-color:#c5daf6; color:#000000;" | 1.22
| style="background-color:#c5daf6; color:#000000;" | 94
| style="background-color:#c5daf6; color:#000000;" | 9.4
| style="background-color:#c5daf6; color:#000000;" | 20
|}
 
{| class="wikitable" style="display: inline-table;"
! style="font-weight:bold; background-color:#efefef; color:#000000;" | Jetson Xavier (30W)
! colspan="2" style="text-align: center; font-weight:bold; background-color:#efefef; color:#000000;" | CPU Library
! colspan="2" style="text-align: center; font-weight:bold; background-color:#efefef; color:#000000;" | GPU Library
|-
| style="background-color:#6aa758; color:#000000;" | Model
| style="background-color:#6aa758; color:#000000;" | Framerate
| style="background-color:#6aa758; color:#000000;" | CPU Usage
| style="background-color:#6aa758; color:#000000;" | Framerate
| style="background-color:#6aa758; color:#000000;" | CPU Usage
|-
| style="background-color:#6aa758; color:#000000;" | Inception V1
| style="background-color:#d8e9d3; color:#000000;" | 6.41
| style="background-color:#d8e9d3; color:#000000;" | 93
| style="background-color:#d8e9d3; color:#000000;" | 66.27
| style="background-color:#d8e9d3; color:#000000;" | 72
|-
| style="background-color:#6aa758; color:#000000;" | Inception V2
| style="background-color:#d8e9d3; color:#000000;" | 5.11
| style="background-color:#d8e9d3; color:#000000;" | 95
| style="background-color:#d8e9d3; color:#000000;" | 50.59
| style="background-color:#d8e9d3; color:#000000;" | 62
|-
| style="background-color:#6aa758; color:#000000;" | Inception V3
| style="background-color:#d8e9d3; color:#000000;" | 1.96
| style="background-color:#d8e9d3; color:#000000;" | 98
| style="background-color:#d8e9d3; color:#000000;" | 22.95
| style="background-color:#d8e9d3; color:#000000;" | 44
|-
| style="background-color:#6aa758; color:#000000;" | Inception V4
| style="background-color:#d8e9d3; color:#000000;" | 0.98
| style="background-color:#d8e9d3; color:#000000;" | 99
| style="background-color:#d8e9d3; color:#000000;" | 12.14
| style="background-color:#d8e9d3; color:#000000;" | 32
|}
 
=== Framerate ===
 
[[File:Framerate Benchmarks gst-inference.png|1024px|frameless|thumb|center]]
 
=== CPU Usage ===
 
[[File:CPU Benchmarks gst-inference.png|1024px|frameless|thumb|center]]
 
== TensorFlow Lite Benchmarks ==
=== FPS measurement ===
<html>
  <style>
    .button {
    background-color: #008CBA;
    border: none;
    color: white;
    padding: 15px 32px;
    text-align: center;
    text-decoration: none;
    display: inline-block;
    font-size: 16px;
    margin: 4px 2px;
    cursor: pointer;
  }
</style>
  <div id="Buttons_Model" style="margin: auto; width: 1300px; height: auto;">
  <button class="button" id="show_inceptionv1">Show InceptionV1 </button>
  <button class="button" id="show_inceptionv2">Show InceptionV2 </button>
  <button class="button" id="show_inceptionv3">Show InceptionV3 </button>
  <button class="button" id="show_inceptionv4">Show InceptionV4 </button>
  <button class="button" id="show_tinyyolov2">Show TinyYoloV2 </button>
  <button class="button" id="show_tinyyolov3">Show TinyYoloV3 </button>
</div>
<br><br>
<div id="chart_div" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<div id="Buttons_Backend" style="margin: auto; width: 600px; height: auto;">
<button class="button" id="show_tensorflow">Show Tensorflow </button>
<button class="button" id="show_tflite">Show Tensorflow-Lite </button>
<button class="button" id="show_edgetpu">Show EdgeTPU </button>
</div>
<div id="chart_div1" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<div id="Buttons_Platform" style="margin: auto; width: 1050px; height: auto;">
<button class="button" id="show_x86">Show x86 </button>
<button class="button" id="show_x86_gpu">Show X86+GPU </button>
<button class="button" id="show_tx2">Show Tx2 </button>
<button class="button" id="show_tx2_gpu">Show TX2+GPU </button>
<button class="button" id="show_coral">Show Coral </button>
<button class="button" id="show_coral_tpu">Show Coral+TPU </button>
</div>
<div id="chart_div2" style="margin: auto; width: 800px; height: 500px;"></div>
 
<script>
      google.charts.load('current', {'packages':['corechart', 'bar']});
      google.charts.setOnLoadCallback(drawStuff);
     
 
      function drawStuff() {
 
 
 
        var chartDiv = document.getElementById('chart_div');
        var chartDiv1 = document.getElementById('chart_div1');
        var chartDiv2 = document.getElementById('chart_div2');
 
        var table_backend_platform_fps = google.visualization.arrayToDataTable([
          ['Platform',                      //Colunm 0
          'InceptionV1 \n Tensorflow',     //Colunm 1
          'InceptionV1 \n Tensorflow-Lite',//Colunm 2
          'InceptionV1 \n EdgeTPU',        //Colunm 3
          'InceptionV2 \n Tensorflow',    //Colunm 4
          'InceptionV2 \n Tensorflow-Lite',//Colunm 5
          'InceptionV2 \n EdgeTPU',        //Colunm 6
          'InceptionV3 \n Tensorflow',    //Colunm 7
          'InceptionV3 \n Tensorflow-Lite',//Colunm 8
          'InceptionV3 \n EdgeTPU',        //Colunm 9
          'InceptionV4 \n Tensorflow',    //Colunm 10
          'InceptionV4 \n Tensorflow-Lite',//Colunm 11
          'InceptionV4 \n EdgeTPU',        //Colunm 12
          'TinyYoloV2 \n Tensorflow',      //Colunm 13
          'TinyYoloV2 \n Tensorflow-Lite', //Colunm 14
          'TinyYoloV2 \n EdgeTPU',        //Colunm 15
          'TinyYoloV3 \n Tensorflow',      //Colunm 16
          'TinyYoloV3 \n Tensorflow-Lite', //Colunm 17
          'TinyYoloV3 \n EdgeTPU'],        //Colunm 18
          ['x86',      63.747, 22.781, 0, 48.381, 14.164, 0, 20.482, 12.164, 0, 10.338, 10.164, 0, 24.255, 12.164, 0, 27.113, 10.164, 0],
          ['x86+GPU',  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], //row 1
          ['TX2',      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], //row 2
          ['TX2-GPU',  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], //row 3
          ['Coral',    0, 3.107, 0, 0, 2.311, 0, 0, 0.903, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], //row 4
          ['Coral+TPU', 0, 0, 41.489, 0, 0, 42.001, 0, 0, 15.232, 0, 0, 8.405, 0, 0, 0, 0, 0, 0]  //row 5
        ]);
        var table_model_platform_fps = google.visualization.arrayToDataTable([
          ['Model',                        //Colunm 0
            'Tensorflow \n x86',            //Colunm 1
            'Tensorflow \n x86+GPU',        //Colunm 2
            'Tensorflow \n TX2',            //Colunm 3
            'Tensorflow \n TX2+GPU',        //Colunm 4
            'Tensorflow \n Coral',          //Colunm 5
            'Tensorflow \n Coral+TPU',      //Colunm 6
            'Tensorflow-Lite \n x86',      //Colunm 7
            'Tensorflow-Lite \n x86+GPU',  //Colunm 8
            'Tensorflow-Lite \n TX2',      //Colunm 9
            'Tensorflow-Lite \n TX2+GPU',  //Colunm 10
            'Tensorflow-Lite \n Coral',     //Colunm 11
            'Tensorflow-Lite \n Coral+TPU', //Colunm 12
            'EdgeTPU \n Coral+TPU'],        //Colunm 13
          ['InceptionV1', 63.747, 0, 0, 0, 0, 0, 22.781, 0, 0, 0, 3.107, 0, 41.489], //row 1
          ['InceptionV2', 48.381, 0, 0, 0, 0, 0, 14.164, 0, 0, 0, 2.311, 0, 42.001], //row 2
          ['InceptionV3', 20.482, 0, 0, 0, 0, 0, 12.164, 0, 0, 0, 0.903, 0, 15.232], //row 3
          ['InceptionV4', 10.338, 0, 0, 0, 0, 0, 10.164, 0, 0, 0, 0, 0, 8.405], //row 4
          ['TinyYoloV2',  24.255, 0, 0, 0, 0, 0, 12.164, 0, 0, 0, 0, 0, 0], //row 5
          ['TinyYoloV3',  27.113, 0, 0, 0, 0, 0, 10.164, 0, 0, 0, 0, 0, 0]  //row 6
        ]);
        var table_model_backend_fps = google.visualization.arrayToDataTable([
        ['Model',                          //Colunm 0
          'Tensorflow \n x86',            //Colunm 1
          'Tensorflow \n x86+GPU',        //Colunm 2
          'Tensorflow \n TX2',            //Colunm 3
          'Tensorflow \n TX2+GPU',        //Colunm 4
          'Tensorflow \n Coral',          //Colunm 5
          'Tensorflow \n Coral+TPU',      //Colunm 6
          'Tensorflow-Lite \n x86',      //Colunm 7
          'Tensorflow-Lite \n x86+GPU',  //Colunm 8
          'Tensorflow-Lite \n TX2',      //Colunm 9
          'Tensorflow-Lite \n TX2+GPU',  //Colunm 10
          'Tensorflow-Lite \n Coral',     //Colunm 11
          'Tensorflow-Lite \n Coral+TPU', //Colunm 12
          'EdgeTPU \n Coral+TPU'],        //Colunm 13
          ['InceptionV1', 63.747, 0, 0, 0, 0, 0, 22.781, 0, 0, 0, 3.107, 0, 41.489], //row 1
          ['InceptionV2', 48.381, 0, 0, 0, 0, 0, 14.164, 0, 0, 0, 2.311, 0, 42.001], //row 2
          ['InceptionV3', 20.482, 0, 0, 0, 0, 0, 12.164, 0, 0, 0, 0.903, 0, 15.232], //row 3
          ['InceptionV4', 10.338, 0, 0, 0, 0, 0, 10.164, 0, 0, 0, 0, 0, 8.405], //row 4
          ['TinyYoloV2',  24.255, 0, 0, 0, 0, 0, 12.164, 0, 0, 0, 0, 0, 0], //row 5
          ['TinyYoloV3',  27.113, 0, 0, 0, 0, 0, 10.164, 0, 0, 0, 0, 0, 0]  //row 6
        ]);
 
        var materialOptions = {
          width: 900,
          chart: {
            title: 'Model Vs Platform per backend',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'FPS'}, // Left y-axis.
            }
          }
        };
        var materialOptions1 = {
          width: 900,
          chart: {
            title: 'Model Vs Platform per backend',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'FPS'}, // Left y-axis.
            }
          }
        };
        var materialOptions2 = {
          width: 900,
          chart: {
            title: 'Model Vs Backend per Platform',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'FPS'}, // Left y-axis.
            }
          }
        };
 
        var materialChart_fps = new google.charts.Bar(chartDiv);
        var materialChart1_fps = new google.charts.Bar(chartDiv1);
        var materialChart2_fps = new google.charts.Bar(chartDiv2);
        view_fps = new google.visualization.DataView(table_backend_platform_fps);
        view1_fps = new google.visualization.DataView(table_model_platform_fps);
        view2_fps = new google.visualization.DataView(table_model_backend_fps);
 
        function drawMaterialChart() {
          var materialChart_fps = new google.charts.Bar(chartDiv);
          var materialChart1_fps = new google.charts.Bar(chartDiv1);
          var materialChart2_fps = new google.charts.Bar(chartDiv2);
          materialChart_fps.draw(table_backend_platform_fps, google.charts.Bar.convertOptions(materialOptions));
          materialChart1_fps.draw(table_model_platform_fps, google.charts.Bar.convertOptions(materialOptions1));
          materialChart2_fps.draw(table_model_backend_fps, google.charts.Bar.convertOptions(materialOptions2));
          init_charts();
        }
        function init_charts(){
          view_fps.setColumns([0,1,2,3]);
          view_fps.hideColumns([4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]);
          materialChart_fps.draw(view_fps, materialOptions);
          view1_fps.setColumns([0,1,2,3,4,5,6]);
          view1_fps.hideColumns([7,8,9,10,11,12]);
          materialChart1_fps.draw(view1_fps, materialOptions1);
          view2_fps.setColumns([0,1,7]);
          view2_fps.hideColumns([2,3,4,5,6,8,9,10,11,12]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
 
        }
 
        // REF_MODEL
        /*Select the Model that you want to show in the chart*/
        var show_inceptionv1 = document.getElementById('show_inceptionv1');
        show_inceptionv1.onclick = function () {
          view_fps.setColumns([0,1]);
          view_fps.hideColumns([2, 3, 4, 5, 6]);
          materialChart_fps.draw(view_fps, materialOptions);
        }
        var show_inceptionv2 = document.getElementById("show_inceptionv2");
        show_inceptionv2.onclick = function() {
          view_fps.setColumns([0,4,5,6]);
          view_fps.hideColumns([1,2,3,7,8,9,10,11,12,13,14,15,16,17,18]);
          materialChart_fps.draw(view_fps, materialOptions);
        }
        var show_inceptionv3 = document.getElementById('show_inceptionv3');
        show_inceptionv3.onclick = function() {
          view_fps.setColumns([0,7,8,9]);
          view_fps.hideColumns([1,2,3,4,5,6,10,11,12,13,14,15,16,17,18]);
          materialChart_fps.draw(view_fps, materialOptions);
        }
        var show_inceptionv4 = document.getElementById("show_inceptionv4");
        show_inceptionv4.onclick = function() {
          view_fps.setColumns([0,10,11,12]);
          view_fps.hideColumns([1,2,3,4,5,6,7,8,9,13,14,15,16,17,18]);
          materialChart_fps.draw(view_fps, materialOptions);
        }
        var show_tinyyolov2 = document.getElementById("show_tinyyolov2");
        show_tinyyolov2.onclick = function() {
          view_fps.setColumns([0,13,14,15]);
          view_fps.hideColumns([1,2,3,4,5,6,7,8,9,10,11,12,16,17,18]);
          materialChart_fps.draw(view_fps, materialOptions);
        }
        var show_tinyyolov3 = document.getElementById("show_tinyyolov3");
        show_tinyyolov3.onclick = function() {
          view_fps.setColumns([0,16,17,18]);
          view_fps.hideColumns([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]);
          materialChart_fps.draw(view_fps, materialOptions);
        }
        /*  REF_BACKEND  */
        /*Select the backend to filter the data to show in the chart*/
        var button = document.getElementById('show_tensorflow');
        button.onclick = function() {
          view1_fps.setColumns([0,1,2,3,4,5,6]);
          view1_fps.hideColumns([7,8,9,10,11,12,13]);
          materialChart1_fps.draw(view1_fps, materialOptions1);
        }
        var showFPS_only = document.getElementById("show_tflite");
        showFPS_only.onclick = function() {
          view1_fps.setColumns([0,7,8,9,10,11,12]);
          view1_fps.hideColumns([1,2,3,4,5,6,13]);
          materialChart1_fps.draw(view1_fps, materialOptions1);
        }
        var showEdgeTPUFPS = document.getElementById("show_edgetpu");
        showEdgeTPUFPS.onclick = function() {
          view1_fps.setColumns([0,13]);
          view1_fps.hideColumns([1,2,3,4,5,6,7,8,9,10,11,12]);
          materialChart1_fps.draw(view1_fps, materialOptions1);
        }
        /*  REF_PLATFORM  */
        /*Select the Platform to filter the data to show in the chart*/
        var show_x86 = document.getElementById('show_x86');
        show_x86.onclick = function() {
          view2_fps.setColumns([0,1,7]);
          view2_fps.hideColumns([2,3,4,5,6,8,9,10,11,12]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
        }
        var show_x86_gpu = document.getElementById("show_x86_gpu");
        show_x86_gpu.onclick = function() {
          view2_fps.setColumns([0,2,8]);
          view2_fps.hideColumns([1,3,4,5,6,7,9,10,11,12]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
        }
        var show_tx2 = document.getElementById('show_tx2');
        show_tx2.onclick = function() {
          view2_fps.setColumns([0,3,9]);
          view2_fps.hideColumns([1,2,4,5,6,7,8,10,11,12]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
        }
        var show_tx2_gpu = document.getElementById("show_tx2_gpu");
        show_tx2_gpu.onclick = function() {
          view2_fps.setColumns([0,4,10]);
          view2_fps.hideColumns([1,2,3,5,6,7,8,9,11,12]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
        }
        var show_coral = document.getElementById('show_coral');
        show_coral.onclick = function() {
          view2_fps.setColumns([0,5,11]);
          view2_fps.hideColumns([1,2,3,4,6,7,8,9,10,12]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
        }
        var show_coral_tpu = document.getElementById("show_coral_tpu");
        show_coral_tpu.onclick = function() {
          view2_fps.setColumns([0,13]);
          view2_fps.hideColumns([1,2,3,4,5,7,8,9,10,11]);
          materialChart2_fps.draw(view2_fps, materialOptions2);
        }
 
        drawMaterialChart();
    };
</script>
</html>
 
=== CPU usage measurement ===
<html>
 
<div id="Buttons_Model" style="margin: auto; width: 1300px; height: auto;">
<button class="button" id="show_inceptionv1_cpu">Show InceptionV1 </button>
<button class="button" id="show_inceptionv2_cpu">Show InceptionV2 </button>
<button class="button" id="show_inceptionv3_cpu">Show InceptionV3 </button>
<button class="button" id="show_inceptionv4_cpu">Show InceptionV4 </button>
<button class="button" id="show_tinyyolov2_cpu">Show TinyYoloV2 </button>
<button class="button" id="show_tinyyolov3_cpu">Show TinyYoloV3 </button>
</div>
<br><br>
<div id="chart_div_cpu" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<div id="Buttons_Backend" style="margin: auto; width: 600px; height: auto;">
<button class="button" id="show_tensorflow_cpu">Show Tensorflow </button>
<button class="button" id="show_tflite_cpu">Show Tensorflow-Lite </button>
<button class="button" id="show_edgetpu_cpu">Show EdgeTPU </button>
</div>
<div id="chart_div1_cpu" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<div id="Buttons_Platform" style="margin: auto; width: 1050px; height: auto;">
<button class="button" id="show_x86_cpu">Show x86 </button>
<button class="button" id="show_x86_gpu_cpu">Show X86+GPU </button>
<button class="button" id="show_tx2_cpu">Show Tx2 </button>
<button class="button" id="show_tx2_gpu_cpu">Show TX2+GPU </button>
<button class="button" id="show_coral_cpu">Show Coral </button>
<button class="button" id="show_coral_tpu_cpu">Show Coral+TPU </button>
</div>
<div id="chart_div2_cpu" style="margin: auto; width: 800px; height: 500px;"></div>
 
<script>
            google.charts.load('current', {'packages':['corechart', 'bar']});
      google.charts.setOnLoadCallback(drawStuff1);
     
 
      function drawStuff1() {
 
 
 
        var chartDivCPU = document.getElementById('chart_div_cpu');
        var chartDiv1CPU = document.getElementById('chart_div1_cpu');
        var chartDiv2CPU = document.getElementById('chart_div2_cpu');
 
        var table_backend_platform = google.visualization.arrayToDataTable([
          ['Platform',
          'InceptionV1 \n Tensorflow',
          'InceptionV1 \n Tensorflow-Lite',
          'InceptionV1 \n EdgeTPU',
          'InceptionV2 \n Tensorflow',
          'InceptionV2 \n Tensorflow-Lite',
          'InceptionV2 \n EdgeTPU',
          'InceptionV3 \n Tensorflow',
          'InceptionV3 \n Tensorflow-Lite',
          'InceptionV3 \n EdgeTPU',
          'InceptionV4 \n Tensorflow',
          'InceptionV4 \n Tensorflow-Lite',
          'InceptionV4 \n EdgeTPU',
          'TinyYoloV2 \n Tensorflow',
          'TinyYoloV2 \n Tensorflow-Lite',
          'TinyYoloV2 \n EdgeTPU',
          'TinyYoloV3 \n Tensorflow',
          'TinyYoloV3 \n Tensorflow-Lite',
          'TinyYoloV3 \n EdgeTPU'],
          ['x86',      74, 46, 0, 75, 43, 0, 79, 54, 0, 84, 50, 0, 79, 45, 0, 76, 44, 0],
          ['x86+GPU',  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
          ['TX2',      0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
          ['TX2-GPU',  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
          ['Coral',     0, 73, 0, 0, 72, 0, 0, 74, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
          ['Coral+TPU', 0, 0, 32, 0, 0, 32, 0, 0, 12, 0, 0, 6, 0, 0, 0, 0, 0, 0]
        ]);
        var table_model_platform = google.visualization.arrayToDataTable([
          ['Model',
          'Tensorflow \n x86',
          'Tensorflow \n x86+GPU',
          'Tensorflow \n TX2',
          'Tensorflow \n TX2+GPU',
          'Tensorflow \n Coral',
          'Tensorflow \n Coral+TPU',
          'Tensorflow-Lite \n x86',
          'Tensorflow-Lite \n x86+GPU',
          'Tensorflow-Lite \n TX2',
          'Tensorflow-Lite \n TX2+GPU',
          'Tensorflow-Lite \n Coral',
          'Tensorflow-Lite \n Coral+TPU',
          'EdgeTPU \n Coral+TPU'],
          ['InceptionV1', 74, 0, 0, 0, 0, 0, 46, 0, 0, 0, 73, 0, 32],
          ['InceptionV2', 75, 0, 0, 0, 0, 0, 43, 0, 0, 0, 72, 0, 32],
          ['InceptionV3', 79, 0, 0, 0, 0, 0, 54, 0, 0, 0, 74, 0, 12],
          ['InceptionV4', 84, 0, 0, 0, 0, 0, 50, 0, 0, 0, 0, 0, 6],
          ['TinyYoloV2',  79, 0, 0, 0, 0, 0, 45, 0, 0, 0, 0, 0, 0],
          ['TinyYoloV3',  76, 0, 0, 0, 0, 0, 44, 0, 0, 0, 0, 0, 0]
        ]);
        var table_model_backend = google.visualization.arrayToDataTable([
        ['Model',
          'Tensorflow \n x86',
          'Tensorflow \n x86+GPU',
          'Tensorflow \n TX2',
          'Tensorflow \n TX2+GPU',
          'Tensorflow \n Coral',
          'Tensorflow \n Coral+TPU',
          'Tensorflow-Lite \n x86',
          'Tensorflow-Lite \n x86+GPU',
          'Tensorflow-Lite \n TX2',
          'Tensorflow-Lite \n TX2+GPU',
          'Tensorflow-Lite \n Coral',
          'Tensorflow-Lite \n Coral+TPU',
          'EdgeTPU \n Coral+TPU'],
          ['InceptionV1', 74, 0, 0, 0, 0, 0, 46, 0, 0, 0, 73, 0, 32],
          ['InceptionV2', 75, 0, 0, 0, 0, 0, 43, 0, 0, 0, 72, 0, 32],
          ['InceptionV3', 79, 0, 0, 0, 0, 0, 54, 0, 0, 0, 74, 0, 12],
          ['InceptionV4', 84, 0, 0, 0, 0, 0, 50, 0, 0, 0, 0, 0, 6],
          ['TinyYoloV2',  79, 0, 0, 0, 0, 0, 45, 0, 0, 0, 0, 0, 0],
          ['TinyYoloV3',  76, 0, 0, 0, 0, 0, 44, 0, 0, 0, 0, 0, 0]
        ]);
 
        var materialOptions = {
          width: 900,
          chart: {
            title: 'Backend Vs Platform per model',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'CPU USAGE'}, // Left y-axis.
            }
          }
        };
        var materialOptions1 = {
          width: 900,
          chart: {
            title: 'Model Vs Platform per backend',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'CPU USAGE'}, // Left y-axis.
            }
          }
        };
        var materialOptions2 = {
          width: 900,
          chart: {
            title: 'Model Vs Backend per Platform',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'CPU USAGE'}, // Left y-axis.
            }
          }
        };
 
        var materialChart = new google.charts.Bar(chartDivCPU);
        var materialChart1 = new google.charts.Bar(chartDiv1CPU);
        var materialChart2 = new google.charts.Bar(chartDiv2CPU);
        view = new google.visualization.DataView(table_backend_platform);
        view1 = new google.visualization.DataView(table_model_platform);
        view2 = new google.visualization.DataView(table_model_backend);
 
        function drawMaterialChart() {
          var materialChart = new google.charts.Bar(chartDivCPU);
          var materialChart1 = new google.charts.Bar(chartDiv1CPU);
          var materialChart2 = new google.charts.Bar(chartDiv2CPU);
          materialChart.draw(table_backend_platform, google.charts.Bar.convertOptions(materialOptions));
          materialChart1.draw(table_model_platform, google.charts.Bar.convertOptions(materialOptions1));
          materialChart2.draw(table_model_backend, google.charts.Bar.convertOptions(materialOptions2));
          init_charts();
        }
        function init_charts(){
          view.setColumns([0,1,2,3]);
          view.hideColumns([4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]);
          materialChart.draw(view, materialOptions);
          view1.setColumns([0,1,2,3,4,5,6]);
          view1.hideColumns([7,8,9,10,11,12]);
          materialChart1.draw(view1, materialOptions1);
          view2.setColumns([0,1,7]);
          view2.hideColumns([2,3,4,5,6,8,9,10,11,12]);
          materialChart2.draw(view2, materialOptions2);
 
        }
 
 
        // REF_MODEL CPU
        /*Select the Model that you want to show in the chart*/
        var show_inceptionv1_cpu = document.getElementById('show_inceptionv1_cpu');
        show_inceptionv1_cpu.onclick = function () {
          view.setColumns([0,1,2,3]);
          view.hideColumns([4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]);
          materialChart.draw(view, materialOptions);
        }
        var show_inceptionv2_cpu = document.getElementById("show_inceptionv2_cpu");
        show_inceptionv2_cpu.onclick = function() {
          view.setColumns([0,4,5,6]);
          view.hideColumns([1,2,3,7,8,9,10,11,12,13,14,15,16,17,18]);
          materialChart.draw(view, materialOptions);
        }
        var show_inceptionv3_cpu = document.getElementById('show_inceptionv3_cpu');
        show_inceptionv3_cpu.onclick = function() {
          view.setColumns([0,7,8,9]);
          view.hideColumns([1,2,3,4,5,6,10,11,12,13,14,15,16,17,18]);
          materialChart.draw(view, materialOptions);
        }
        var show_inceptionv4_cpu = document.getElementById("show_inceptionv4_cpu");
        show_inceptionv4_cpu.onclick = function() {
          view.setColumns([0,10,11,12]);
          view.hideColumns([1,2,3,4,5,6,7,8,9,13,14,15,16,17,18]);
          materialChart.draw(view, materialOptions);
        }
        var show_tinyyolov2_cpu = document.getElementById("show_tinyyolov2_cpu");
        show_tinyyolov2_cpu.onclick = function() {
          view.setColumns([0,13,14,15]);
          view.hideColumns([1,2,3,4,5,6,7,8,9,10,11,12,16,17,18]);
          materialChart.draw(view, materialOptions);
        }
        var show_tinyyolov3_cpu = document.getElementById("show_tinyyolov3_cpu");
        show_tinyyolov3_cpu.onclick = function() {
          view.setColumns([0,16,17,18]);
          view.hideColumns([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]);
          materialChart.draw(view, materialOptions);
        }
        /*  REF_BACKEND  CPU*/
        /*Select the backend to filter the data to show in the chart*/
        var button = document.getElementById('show_tensorflow_cpu');
        button.onclick = function() {
          view1.setColumns([0,1,2,3,4,5,6]);
          view1.hideColumns([7,8,9,10,11,12,13]);
          materialChart1.draw(view1, materialOptions1);
        }
        var showCPU_USAGE_only = document.getElementById("show_tflite_cpu");
        showCPU_USAGE_only.onclick = function() {
          view1.setColumns([0,7,8,9,10,11,12]);
          view1.hideColumns([1,2,3,4,5,6,13]);
          materialChart1.draw(view1, materialOptions1);
        }
        var showCPU_USAGE_EdgeTPU = document.getElementById("show_edgetpu_cpu");
        showCPU_USAGE_EdgeTPU.onclick = function() {
          view1.setColumns([0,13]);
          view1.hideColumns([1,2,3,4,5,6,7,8,9,10,11,12]);
          materialChart1.draw(view1, materialOptions1);
        }
        /*  REF_PLATFORM  CPU*/
        /*Select the Platform to filter the data to show in the chart*/
        var show_x86_cpu = document.getElementById('show_x86_cpu');
        show_x86_cpu.onclick = function() {
          view2.setColumns([0,1,7]);
          view2.hideColumns([2,3,4,5,6,8,9,10,11,12,13]);
          materialChart2.draw(view2, materialOptions2);
        }
        var show_x86_gpu_cpu = document.getElementById("show_x86_gpu_cpu");
        show_x86_gpu_cpu.onclick = function() {
          view2.setColumns([0,2,8]);
          view2.hideColumns([1,3,4,5,6,7,9,10,11,12,13]);
          materialChart2.draw(view2, materialOptions2);
        }
        var show_tx2_cpu = document.getElementById('show_tx2_cpu');
        show_tx2_cpu.onclick = function() {
          view2.setColumns([0,3,9]);
          view2.hideColumns([1,2,4,5,6,7,8,10,11,12,13]);
          materialChart2.draw(view2, materialOptions2);
        }
        var show_tx2_gpu_cpu = document.getElementById("show_tx2_gpu_cpu");
        show_tx2_gpu_cpu.onclick = function() {
          view2.setColumns([0,4,10]);
          view2.hideColumns([1,2,3,5,6,7,8,9,11,12,13]);
          materialChart2.draw(view2, materialOptions2);
        }
        var show_coral_cpu = document.getElementById('show_coral_cpu');
        show_coral_cpu.onclick = function() {
          view2.setColumns([0,5,11]);
          view2.hideColumns([1,2,3,4,6,7,8,9,10,12,13]);
          materialChart2.draw(view2, materialOptions2);
        }
        var show_coral_tpu_cpu = document.getElementById("show_coral_tpu_cpu");
        show_coral_tpu_cpu.onclick = function() {
          view2.setColumns([0,6,12,13]);
          view2.hideColumns([1,2,3,4,5,7,8,9,10,11]);
          materialChart2.draw(view2, materialOptions2);
        }
 
        drawMaterialChart();
    };
</script>
</html>


=== Test benchmark video ===
=== Test benchmark video ===
The following video was used to perform the benchmark tests.
The following video was used to perform the benchmark tests.
<br>
<br>
To download the video press right click on the video and select 'Save video as' and save this in your computer.
To download the video press right-click on the video and select 'Save video as' and save this on your computer.
<br>
<br>
[[File:Test benchmark video.mp4|thumb|border|center|500px|alt=Alt|Video 1. Test benchmark video]]


[[File:Test benchmark video.mp4|500px|thumb|center|Test benchmark video]]
== x86 ==


== ONNXRT Benchmarks ==
The Desktop PC had the following specifications:
The Desktop PC had the following specifications:
*Intel(R) Core(TM) Core i7-7700HQ CPU @ 2.80GHz
*Intel(R) Core(TM) Core i7-7700HQ CPU @ 2.80GHz
*12 GB RAM
*12 GB RAM
*Linux 4.15.0-106-generic x86_64 (Ubuntu 16.04)
*Linux 4.15.0-106-generic x86_64 (Ubuntu 16.04)
*GStreamer 1.8.3
*GStreamer 1.8.3
The following was the GStreamer pipeline used to obtain the results:
<source lang="bash">
# MODELS_PATH has the following structure
#/path/to/models/
#├── InceptionV1_onnxrt
#│   ├── graph_inceptionv1_info.txt
#│   ├── graph_inceptionv1.onnx
#│   └── labels.txt
#├── InceptionV2_onnxrt
#│   ├── graph_inceptionv2_info.txt
#│   ├── graph_inceptionv2.onnx
#│   └── labels.txt
#├── InceptionV3_onnxrt
#│   ├── graph_inceptionv3_info.txt
#│   ├── graph_inceptionv3.onnx
#│   └── labels.txt
#├── InceptionV4_onnxrt
#│   ├── graph_inceptionv4_info.txt
#│   ├── graph_inceptionv4.onnx
#│   └── labels.txt
#├── TinyYoloV2_onnxrt
#│   ├── graph_tinyyolov2_info.txt
#│   ├── graph_tinyyolov2.onnx
#│   └── labels.txt
#└── TinyYoloV3_onnxrt
#    ├── graph_tinyyolov3_info.txt
#    ├── graph_tinyyolov3.onnx
#    └── labels.txt
model_array=(inceptionv1 inceptionv2 inceptionv3 inceptionv4 tinyyolov2 tinyyolov3)
model_upper_array=(InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3)
MODELS_PATH=/path/to/models/
INTERNAL_PATH=onnxrt
EXTENSION=".onnx"
gst-launch-1.0 \
filesrc location=$VIDEO_PATH num-buffers=600 ! decodebin ! videoconvert ! \
perf print-arm-load=true name=inputperf ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue ! net.sink_bypass \
${model_array[i]} backend=onnxrt name=net \
model-location="${MODELS_PATH}${model_upper_array[i]}_${INTERNAL_PATH}/graph_${model_array[i]}${EXTENSION}" \
net.src_bypass ! perf print-arm-load=true name=outputperf ! videoconvert ! fakesink sync=false
</source>


=== FPS Measurements ===
=== FPS Measurements ===
Line 821: Line 75:
</style>
</style>


<div id="Buttons_Model_Onnxrt_Fps" style="margin: auto; width: 1300px; height: auto;">
<div id="chart_fps_x86" style="margin: auto; width: 800px; height: 500px;"></div>
<button class="button" id="show_inceptionv1_onnxrt_fps">Show InceptionV1 </button>
<button class="button" id="show_inceptionv2_onnxrt_fps">Show InceptionV2 </button>
<button class="button" id="show_inceptionv3_onnxrt_fps">Show InceptionV3 </button>
<button class="button" id="show_inceptionv4_onnxrt_fps">Show InceptionV4 </button>
<button class="button" id="show_tinyyolov2_onnxrt_fps">Show TinyYoloV2 </button>
<button class="button" id="show_tinyyolov3_onnxrt_fps">Show TinyYoloV3 </button>
</div>
<br><br>
<div id="chart_onnxrt" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<div id="Buttons_Backend_Onnxrt_Fps" style="margin: auto; width: 600px; height: auto;">
<button class="button" id="show_onnxrt_fps">Show ONNXRT </button>
</div>
<div id="chart_onnxrt1" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>


<script>
<script>
       google.charts.load('current', {'packages':['corechart', 'bar']});
       google.charts.load('current', {'packages':['corechart', 'bar']});
       google.charts.setOnLoadCallback(drawStuffFpsOnnxrt);
       google.charts.setOnLoadCallback(drawStuffx86Fps);
        
        
       function drawStuffFpsOnnxrt() {
       function drawStuffx86Fps() {


         var onnxrt_chartDiv = document.getElementById('chart_onnxrt');
         var chartDiv_Fps_x86 = document.getElementById('chart_fps_x86');
        var onnxrt_chartDiv1 = document.getElementById('chart_onnxrt1');


         var table_backend_platform_onnxrt_fps = google.visualization.arrayToDataTable([
         var table_models_fps_x86 = google.visualization.arrayToDataTable([
           ['Platform',                 //Column 0
           ['Model',                     //Column 0
           'InceptionV1 \n ONNXRT',     //Column 1
           'ONNXRT \n x86',
           'InceptionV2 \n ONNXRT',     //Column 2
          'ONNXRT OpenVINO (CPU_FP32) \n x86',
           'InceptionV3 \n ONNXRT',     //Column 3
           'ONNXRT OpenVINO (GPU_FP32)\n x86',
           'InceptionV4 \n ONNXRT',     //Column 4
           'ONNXRT OpenVINO (GPU_FP16)\n x86',
           'TinyYoloV2 \n ONNXRT',     //Column 5
           'ONNXRT OpenVINO (MYRIAD_FP16)\n x86',
           'TinyYoloV3 \n ONNXRT'],     //Column 6
           'TensorFlow \n x86',
           ['x86', 47.8702, 32.7236, 12.092, 5.2632, 16.03, 18.3592]
           'TensorFlow Lite \n x86'],
        ]);
           ['InceptionV1', 47.9, 81.966, 70.580, 98.742, 46.294, 55.3182, 18.8422], //row 1
        var table_model_platform_onnxrt_fps = google.visualization.arrayToDataTable([
           ['InceptionV2', 32.7, 63.352, 54.159, 77.449, 34.613, 39.6438, 13.5714], //row 2
          ['Model',                 //Column 0
           ['InceptionV3', 12.1, 23.287, 20.878, 34.059, 11.999, 16.2488, 4.9924], //row 3
            'ONNXRT \n x86'],        //Column 1
           ['InceptionV4', 5.26, 10.927, 6.160, 4.548, 6.494, 7.793, 2.583], //row 4
          ['InceptionV1', 47.8702], //row 1
           ['TinyYoloV2', 33.559, 32.587, 0, 0, 0, 18.1846, 7.2708], //row 5
           ['InceptionV2', 32.7236], //row 2
           ['TinyYoloV3', 35.092, 27.799, 0, 0, 0, 21.7334, 7.3042]  //row 6
           ['InceptionV3', 12.092], //row 3
           ['InceptionV4', 5.2632], //row 4
           ['TinyYoloV2', 16.03], //row 5
           ['TinyYoloV3', 18.3592]  //row 6
         ]);
         ]);
         var onnxrt_materialOptions = {
         var x86_materialOptions_fps = {
           width: 320,
           width: 1000,
           chart: {
           chart: {
             title: 'Backend Vs Platform per model',
             title: 'Model Vs FPS per backend',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'FPS'}, // Left y-axis.
            }
          }
        };
        var onnxrt_materialOptions1 = {
          width: 900,
          chart: {
            title: 'Model Vs Platform per backend',
           },
           },
           series: {
           series: {
Line 894: Line 115:
         };
         };


         var materialChart_onnxrt_fps = new google.charts.Bar(onnxrt_chartDiv);
         var materialChart_x86_fps = new google.charts.Bar(chartDiv_Fps_x86);
         var materialChart_onnxrt_fps1 = new google.charts.Bar(onnxrt_chartDiv1);
         view_x86_fps = new google.visualization.DataView(table_models_fps_x86);
        view_onnxrt_fps = new google.visualization.DataView(table_backend_platform_onnxrt_fps);
        view_onnxrt_fps1 = new google.visualization.DataView(table_model_platform_onnxrt_fps);


         function drawMaterialChart() {
         function drawMaterialChart() {
           var materialChart_onnxrt_fps = new google.charts.Bar(onnxrt_chartDiv);
           var materialChart_x86_fps = new google.charts.Bar(chartDiv_Fps_x86);
           var materialChart_onnxrt_fps1 = new google.charts.Bar(onnxrt_chartDiv1);
           materialChart_x86_fps.draw(table_models_fps_x86, google.charts.Bar.convertOptions(x86_materialOptions_fps));
          materialChart_onnxrt_fps.draw(table_backend_platform_onnxrt_fps, google.charts.Bar.convertOptions(onnxrt_materialOptions));
          materialChart_onnxrt_fps1.draw(table_model_platform_onnxrt_fps, google.charts.Bar.convertOptions(onnxrt_materialOptions1));


           init_charts();
           init_charts();
         }
         }
         function init_charts(){
         function init_charts(){
           view_onnxrt_fps.setColumns([0,1]);
           view_x86_fps.setColumns([0,1, 2, 3, 4, 5, 6, 7]);
          view_onnxrt_fps.hideColumns([2, 3, 4, 5, 6]);
           materialChart_x86_fps.draw(view_x86_fps, x86_materialOptions_fps);
           materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
          view_onnxrt_fps1.setColumns([0,1]);
        drawMaterialChart();
          materialChart_onnxrt_fps1.draw(view_onnxrt_fps1, onnxrt_materialOptions1);
         }
         }


        // REF_MODEL
        /*Select the Model that you want to show in the chart*/
        var show_inceptionv1_onnxrt_fps = document.getElementById('show_inceptionv1_onnxrt_fps');
        show_inceptionv1_onnxrt_fps.onclick = function () {
          view_onnxrt_fps.setColumns([0,1]);
          view_onnxrt_fps.hideColumns([2, 3, 4, 5, 6]);
          materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
        var show_inceptionv2_onnxrt_fps = document.getElementById('show_inceptionv2_onnxrt_fps');
        show_inceptionv2_onnxrt_fps.onclick = function () {
          view_onnxrt_fps.setColumns([0,2]);
          view_onnxrt_fps.hideColumns([1, 3, 4, 5, 6]);
          materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
        var show_inceptionv3_onnxrt_fps = document.getElementById('show_inceptionv3_onnxrt_fps');
        show_inceptionv3_onnxrt_fps.onclick = function () {
          view_onnxrt_fps.setColumns([0,3]);
          view_onnxrt_fps.hideColumns([1, 2, 4, 5, 6]);
          materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
        var show_inceptionv4_onnxrt_fps = document.getElementById('show_inceptionv4_onnxrt_fps');
        show_inceptionv4_onnxrt_fps.onclick = function () {
          view_onnxrt_fps.setColumns([0,4]);
          view_onnxrt_fps.hideColumns([1, 2, 3, 5, 6]);
          materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
        var show_tinyyolov2_onnxrt_fps = document.getElementById('show_tinyyolov2_onnxrt_fps');
        show_tinyyolov2_onnxrt_fps.onclick = function () {
          view_onnxrt_fps.setColumns([0,5]);
          view_onnxrt_fps.hideColumns([1, 2, 3, 4, 6]);
          materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
        var show_tinyyolov3_onnxrt_fps = document.getElementById('show_tinyyolov3_onnxrt_fps');
        show_tinyyolov3_onnxrt_fps.onclick = function () {
          view_onnxrt_fps.setColumns([0,6]);
          view_onnxrt_fps.hideColumns([1, 2, 3, 4, 5]);
          materialChart_onnxrt_fps.draw(view_onnxrt_fps, onnxrt_materialOptions);
        }
        // REF_BACKEND
        /*Select the Model that you want to show in the chart*/
        var show_onnxrt_fps = document.getElementById('show_onnxrt_fps');
        show_onnxrt_fps.onclick = function () {
          view_onnxrt_fps1.setColumns([0,1]);
          materialChart_onnxrt_fps1.draw(view_onnxrt_fps1, onnxrt_materialOptions1);
        }
        drawMaterialChart();
    };
</script>
</script>


Line 985: Line 154:
</style>
</style>


<div id="Buttons_Model_Onnxrt_Cpu" style="margin: auto; width: 1300px; height: auto;">
<div id="chart_cpu_x86" style="margin: auto; width: 800px; height: 500px;"></div>
<button class="button" id="show_inceptionv1_onnxrt_cpu">Show InceptionV1 </button>
<button class="button" id="show_inceptionv2_onnxrt_cpu">Show InceptionV2 </button>
<button class="button" id="show_inceptionv3_onnxrt_cpu">Show InceptionV3 </button>
<button class="button" id="show_inceptionv4_onnxrt_cpu">Show InceptionV4 </button>
<button class="button" id="show_tinyyolov2_onnxrt_cpu">Show TinyYoloV2 </button>
<button class="button" id="show_tinyyolov3_onnxrt_cpu">Show TinyYoloV3 </button>
</div>
<br><br>
<div id="chart_onnxrt_cpu" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>
<div id="Buttons_Backend_Onnxrt_Cpu" style="margin: auto; width: 600px; height: auto;">
<button class="button" id="show_onnxrt_cpu">Show ONNXRT </button>
</div>
<div id="chart_onnxrt1_cpu" style="margin: auto; width: 800px; height: 500px;"></div>
<br><br>


<script>
<script>
       google.charts.load('current', {'packages':['corechart', 'bar']});
       google.charts.load('current', {'packages':['corechart', 'bar']});
       google.charts.setOnLoadCallback(drawStuffCpuOnnxrt);
       google.charts.setOnLoadCallback(drawStuffx86Cpu);
        
        
       function drawStuffCpuOnnxrt() {
       function drawStuffx86Cpu() {


         var onnxrt_chartDiv_cpu = document.getElementById('chart_onnxrt_cpu');
         var chartDiv_Cpu_x86 = document.getElementById('chart_cpu_x86');
        var onnxrt_chartDiv1_cpu = document.getElementById('chart_onnxrt1_cpu');


         var table_backend_platform_onnxrt_cpu = google.visualization.arrayToDataTable([
         var table_models_cpu_x86 = google.visualization.arrayToDataTable([
           ['Platform',                 //Column 0
           ['Model',                     //Column 0
           'InceptionV1 \n ONNXRT',     //Column 1
           'ONNXRT \n x86',
           'InceptionV2 \n ONNXRT',     //Column 2
           'ONNXRT OpenVINO (CPU_FP32) \n x86',
           'InceptionV3 \n ONNXRT',     //Column 3
           'ONNXRT OpenVINO (GPU_FP32)\n x86',
           'InceptionV4 \n ONNXRT',     //Column 4
           'ONNXRT OpenVINO (GPU_FP16)\n x86',
           'TinyYoloV2 \n ONNXRT',     //Column 5
           'ONNXRT OpenVINO (MYRIAD_FP16)\n x86',
           'TinyYoloV3 \n ONNXRT'],    //Column 6
           'TensorFlow \n x86',
          ['x86', 94.6, 100.0, 95.2, 88.8, 94.0, 91.4]
          'TensorFlow Lite \n x86'],        //Column 1
        ]);
           ['InceptionV1', 94.6, 49, 31, 29, 14, 74.2, 47.6], //row 1
        var table_model_platform_onnxrt_cpu = google.visualization.arrayToDataTable([
           ['InceptionV2', 100, 52, 28, 29, 11, 74.2, 43.6], //row 2
          ['Model',                  //Column 0
           ['InceptionV3', 95.2, 49, 28, 28, 13, 81, 60.2], //row 3
            'ONNXRT \n x86'],        //Column 1
           ['InceptionV4', 88.8, 49, 33, 46, 11, 86, 50], //row 4
           ['InceptionV1', 94.6], //row 1
           ['TinyYoloV2',  94, 50, 0, 0, 0, 80.6, 46], //row 5
           ['InceptionV2', 100.0], //row 2
           ['TinyYoloV3',  91.4, 46, 0, 0, 0, 74.6, 42.4]  //row 6
           ['InceptionV3', 95.2], //row 3
           ['InceptionV4', 88.8], //row 4
           ['TinyYoloV2',  94.0], //row 5
           ['TinyYoloV3',  91.4]  //row 6
         ]);
         ]);
         var onnxrt_materialOptions_cpu = {
         var x86_materialOptions_cpu = {
           width: 320,
           width: 1000,
           chart: {
           chart: {
             title: 'Backend Vs Platform per model',
             title: 'Model Vs CPU Load per backend',
           },
           },
           series: {
           series: {
Line 1,040: Line 189:
           axes: {
           axes: {
             y: {
             y: {
               distance: {side: 'left',label: 'CPU Usage'}, // Left y-axis.
               distance: {side: 'left',label: 'CPU Load'}, // Left y-axis.
            }
          }
        };
        var onnxrt_materialOptions1_cpu = {
          width: 900,
          chart: {
            title: 'Model Vs Platform per backend',
          },
          series: {
          },
          axes: {
            y: {
              distance: {side: 'left',label: 'CPU Usage'}, // Left y-axis.
             }
             }
           }
           }
         };
         };


         var materialChart_onnxrt_cpu = new google.charts.Bar(onnxrt_chartDiv_cpu);
         var materialChart_x86_cpu = new google.charts.Bar(chartDiv_Cpu_x86);
         var materialChart_onnxrt_cpu1 = new google.charts.Bar(onnxrt_chartDiv1_cpu);
         view_x86_cpu = new google.visualization.DataView(table_models_cpu_x86);
        view_onnxrt_cpu = new google.visualization.DataView(table_backend_platform_onnxrt_cpu);
        view_onnxrt_cpu1 = new google.visualization.DataView(table_model_platform_onnxrt_cpu);


         function drawMaterialChart() {
         function drawMaterialChart() {
           var materialChart_onnxrt_cpu = new google.charts.Bar(onnxrt_chartDiv_cpu);
           var materialChart_x86_cpu = new google.charts.Bar(chartDiv_Cpu_x86);
           var materialChart_onnxrt_cpu1 = new google.charts.Bar(onnxrt_chartDiv1_cpu);
           materialChart_x86_cpu.draw(table_models_cpu_x86, google.charts.Bar.convertOptions(x86_materialOptions_cpu));
          materialChart_onnxrt_cpu.draw(table_backend_platform_onnxrt_cpu, google.charts.Bar.convertOptions(onnxrt_materialOptions_cpu));
          materialChart_onnxrt_cpu1.draw(table_model_platform_onnxrt_cpu, google.charts.Bar.convertOptions(onnxrt_materialOptions1_cpu));


           init_charts();
           init_charts();
         }
         }
         function init_charts(){
         function init_charts(){
           view_onnxrt_cpu.setColumns([0,1]);
           view_x86_cpu.setColumns([0,1, 2, 3, 4, 5, 6, 7]);
          view_onnxrt_cpu.hideColumns([2, 3, 4, 5, 6]);
           materialChart_x86_cpu.draw(view_x86_cpu, x86_materialOptions_cpu);
           materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
          view_onnxrt_cpu1.setColumns([0,1]);
        drawMaterialChart();
          materialChart_onnxrt_cpu1.draw(view_onnxrt_cpu1, onnxrt_materialOptions1_cpu);
         }
         }


        // REF_MODEL
        /*Select the Model that you want to show in the chart*/
        var show_inceptionv1_onnxrt_cpu = document.getElementById('show_inceptionv1_onnxrt_cpu');
        show_inceptionv1_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu.setColumns([0,1]);
          view_onnxrt_cpu.hideColumns([2, 3, 4, 5, 6]);
          materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
        var show_inceptionv2_onnxrt_cpu = document.getElementById('show_inceptionv2_onnxrt_cpu');
        show_inceptionv2_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu.setColumns([0,2]);
          view_onnxrt_cpu.hideColumns([1, 3, 4, 5, 6]);
          materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
        var show_inceptionv3_onnxrt_cpu = document.getElementById('show_inceptionv3_onnxrt_cpu');
        show_inceptionv3_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu.setColumns([0,3]);
          view_onnxrt_cpu.hideColumns([1, 2, 4, 5, 6]);
          materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
        var show_inceptionv4_onnxrt_cpu = document.getElementById('show_inceptionv4_onnxrt_cpu');
        show_inceptionv4_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu.setColumns([0,4]);
          view_onnxrt_cpu.hideColumns([1, 2, 3, 5, 6]);
          materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
        var show_tinyyolov2_onnxrt_cpu = document.getElementById('show_tinyyolov2_onnxrt_cpu');
        show_tinyyolov2_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu.setColumns([0,5]);
          view_onnxrt_cpu.hideColumns([1, 2, 3, 4, 6]);
          materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
        var show_tinyyolov3_onnxrt_cpu = document.getElementById('show_tinyyolov3_onnxrt_cpu');
        show_tinyyolov3_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu.setColumns([0,6]);
          view_onnxrt_cpu.hideColumns([1, 2, 3, 4, 5]);
          materialChart_onnxrt_cpu.draw(view_onnxrt_cpu, onnxrt_materialOptions_cpu);
        }
        // REF_BACKEND
        /*Select the Model that you want to show in the chart*/
        var show_onnxrt_cpu = document.getElementById('show_onnxrt_cpu');
        show_onnxrt_cpu.onclick = function () {
          view_onnxrt_cpu1.setColumns([0,1]);
          materialChart_onnxrt_cpu1.draw(view_onnxrt_cpu1, onnxrt_materialOptions1);
        }
        drawMaterialChart();
    };
</script>
</script>


</html>
</html>


=== Test benchmark video ===
== Jetson AGX Xavier ==
The following video was used to perform the benchmark tests.
 
<br>
The Jetson Xavier power modes used were 2 and 6 (more information: [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fpower_management_jetson_xavier.html%23wwpID0E0OM0HA Supported Modes and Power Efficiency])
To download the video press  right click on the video and select 'Save video as' and save this in your computer.
 
*View current power mode:


[[File:Test benchmark video.mp4|500px|thumb|center|Test benchmark video]]
<source lang="bash">
$ sudo /usr/sbin/nvpmodel -q
</source>


*Change current power mode:


== Jetson AGX Xavier ==
<source lang="bash">
sudo /usr/sbin/nvpmodel -m x
</source>
Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).


=== FPS Measurements ===
=== FPS Measurements ===
Line 1,175: Line 266:
           'TensorFlow \n Xavier (30 W)',      //Column 3
           'TensorFlow \n Xavier (30 W)',      //Column 3
           'TensorFlow (GPU) \n Xavier (30 W)', //Column 4
           'TensorFlow (GPU) \n Xavier (30 W)', //Column 4
           'TensorRT \n Xavier'],              //Column 5
           'TensorRT \n Xavier',              //Column 5
           ['InceptionV1', 8.24, 52.3, 6.41, 66.27, 92.6], //row 1
          'ONNXRT ACL \n Xavier'],            //Column 6
           ['InceptionV2', 6.58, 39.6, 5.11, 50.59, 0], //row 2
           ['InceptionV1', 8.24, 52.3, 6.41, 66.27, 92.6, 17.566], //row 1
           ['InceptionV3', 2.54, 17.8, 1.96, 22.95, 24.9], //row 3
           ['InceptionV2', 6.58, 39.6, 5.11, 50.59, 0, 12.729], //row 2
           ['InceptionV4', 1.22, 9.4, 0.98, 12.14, 13.6], //row 4
           ['InceptionV3', 2.54, 17.8, 1.96, 22.95, 24.9, 5.709], //row 3
           ['TinyYoloV2',  0, 0, 0, 0, 69.7], //row 5
           ['InceptionV4', 1.22, 9.4, 0.98, 12.14, 13.6, 2.747], //row 4
           ['TinyYoloV3',  0, 0, 0, 0, 0]  //row 6
           ['TinyYoloV2',  0, 0, 0, 0, 69.7, 9.367], //row 5
           ['TinyYoloV3',  0, 0, 0, 0, 0, 10.520]  //row 6
         ]);
         ]);
         var xavier_materialOptions_fps = {
         var xavier_materialOptions_fps = {
Line 1,207: Line 299:
         }
         }
         function init_charts(){
         function init_charts(){
           view_xavier_fps.setColumns([0,1,2,3,4,5]);
           view_xavier_fps.setColumns([0,1,2,3,4,5,6]);
           materialChart_xavier_fps.draw(view_xavier_fps, xavier_materialOptions_fps);
           materialChart_xavier_fps.draw(view_xavier_fps, xavier_materialOptions_fps);
         }
         }
Line 1,252: Line 344:
           'TensorFlow \n Xavier (30 W)',      //Column 3
           'TensorFlow \n Xavier (30 W)',      //Column 3
           'TensorFlow (GPU) \n Xavier (30 W)', //Column 4
           'TensorFlow (GPU) \n Xavier (30 W)', //Column 4
           'TensorRT \n Xavier'],              //Column 5
           'TensorRT \n Xavier',              //Column 5
           ['InceptionV1', 86, 72, 93, 72, 32], //row 1
          'ONNXRT ACL \n Xavier'],            //Column 6
           ['InceptionV2', 88, 62.6, 95, 62, 0], //row 2
           ['InceptionV1', 86, 72, 93, 72, 32, 50], //row 1
           ['InceptionV3', 92, 44, 98, 44, 6], //row 3
           ['InceptionV2', 88, 62.6, 95, 62, 0, 49], //row 2
           ['InceptionV4', 94, 32, 99, 32, 3], //row 4
           ['InceptionV3', 92, 44, 98, 44, 6, 50], //row 3
           ['TinyYoloV2',  0, 0, 0, 0, 16], //row 5
           ['InceptionV4', 94, 32, 99, 32, 3, 50], //row 4
           ['TinyYoloV3',  0, 0, 0, 0, 0]  //row 6
           ['TinyYoloV2',  0, 0, 0, 0, 16, 50], //row 5
           ['TinyYoloV3',  0, 0, 0, 0, 0, 50]  //row 6
         ]);
         ]);
         var xavier_materialOptions_cpu = {
         var xavier_materialOptions_cpu = {
Line 1,284: Line 377:
         }
         }
         function init_charts(){
         function init_charts(){
           view_xavier_cpu.setColumns([0,1,2,3,4,5]);
           view_xavier_cpu.setColumns([0,1,2,3,4,5,6]);
           materialChart_xavier_cpu.draw(view_xavier_cpu, xavier_materialOptions_cpu);
           materialChart_xavier_cpu.draw(view_xavier_cpu, xavier_materialOptions_cpu);
         }
         }
Line 1,591: Line 684:


== Google Coral ==
== Google Coral ==
The following benchmarks were performed on the Coral Dev Board.


=== FPS Measurements ===
=== FPS Measurements ===
Line 1,625: Line 720:
           'TensorFlow Lite \n Coral',
           'TensorFlow Lite \n Coral',
           'TensorFlow Lite EdgeTPU \n Coral'],        //Column 1
           'TensorFlow Lite EdgeTPU \n Coral'],        //Column 1
           ['InceptionV1', 3.11, 41.5], //row 1
           ['InceptionV1', 3.11, 41.6], //row 1
           ['InceptionV2', 2.31, 42], //row 2
           ['InceptionV2', 2.31, 42.8], //row 2
           ['InceptionV3', 0.9, 15.2], //row 3
           ['InceptionV3', 0.9, 15.02], //row 3
           ['InceptionV4', 0, 8.41], //row 4
           ['InceptionV4', 0, 8.56], //row 4
           ['TinyYoloV2',  0, 0], //row 5
           ['MobileNetV2',  0, 41.12], //row 5
           ['TinyYoloV3',  0, 0]  //row 6
           ['MobileNetV2 + SSD',  0, 38.64]  //row 6
         ]);
         ]);
         var Coral_materialOptions_fps = {
         var Coral_materialOptions_fps = {
Line 1,666: Line 761:
</html>
</html>


=== CPU Measurements ===
=== CPU Load Measurements ===


<html>
<html>
Line 1,700: Line 795:
           'TensorFlow Lite EdgeTPU \n Coral'],        //Column 1
           'TensorFlow Lite EdgeTPU \n Coral'],        //Column 1
           ['InceptionV1', 73, 32], //row 1
           ['InceptionV1', 73, 32], //row 1
           ['InceptionV2', 72, 32], //row 2
           ['InceptionV2', 72, 37], //row 2
           ['InceptionV3', 74, 12], //row 3
           ['InceptionV3', 74, 14], //row 3
           ['InceptionV4', 0, 6], //row 4
           ['InceptionV4', 0, 5], //row 4
           ['TinyYoloV2',  0, 0], //row 5
           ['MobileNetV2',  0, 34], //row 5
           ['TinyYoloV3',  0, 0]  //row 6
           ['MobileNetV2 + SSD',  0, 45]  //row 6
         ]);
         ]);
         var Coral_materialOptions_cpu = {
         var Coral_materialOptions_cpu = {
           width: 900,
           width: 900,
           chart: {
           chart: {
             title: 'Model Vs CPU per backend',
             title: 'Model Vs CPU Load per backend',
           },
           },
           series: {
           series: {

Latest revision as of 20:39, 4 September 2024




Previous: Example Applications/DispTec Index Next: Model Zoo


GstInference
GstInference
Introduction
Getting started
Supported architectures

InceptionV1

InceptionV3

YoloV2

AlexNet
Supported backends

Caffe

Metadata and Signals
Overlay Elements
Utils Elements
Legacy pipelines
Example pipelines
Example applications
Benchmarks
Model Zoo
Project Status
Contact Us

Coral from Google




GstInference Benchmarks - Introduction

This wiki summarizes a series of benchmarks on different hardware platforms based on the run_benchmark.sh bash script that can be found in the official GstInference repository. The script is based on the following GStreamer pipeline: 

#Script to run each model
run_all_models(){

  model_array=(inceptionv1 inceptionv2 inceptionv3 inceptionv4 tinyyolov2 tinyyolov3)
  model_upper_array=(InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3)
  input_array=(input input input input input/Placeholder inputs )
  output_array=(InceptionV1/Logits/Predictions/Reshape_1 Softmax InceptionV3/Predictions/Reshape_1 
  InceptionV4/Logits/Predictions add_8 output_boxes )

  mkdir -p logs/
  rm -f logs/*

  for ((i=0;i<${#model_array[@]};++i)); do
    echo Perf ${model_array[i]}
    gst-launch-1.0 \
    filesrc location=$VIDEO_PATH num-buffers=600 ! decodebin ! videoconvert ! \
    perf print-arm-load=true name=inputperf ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue ! net.sink_bypass \
    ${model_array[i]} backend=$BACKEND name=net backend::input-layer=${input_array[i]} backend::output-layer=${output_array[i]} \
    model-location="${MODELS_PATH}${model_upper_array[i]}_${INTERNAL_PATH}/graph_${model_array[i]}${EXTENSION}" \
    net.src_bypass ! perf print-arm-load=true name=outputperf ! videoconvert ! fakesink sync=false > logs/${model_array[i]}.log
  done
}

Test benchmark video

The following video was used to perform the benchmark tests.
To download the video press right-click on the video and select 'Save video as' and save this on your computer.

Video 1. Test benchmark video

x86

The Desktop PC had the following specifications:

  • Intel(R) Core(TM) Core i7-7700HQ CPU @ 2.80GHz
  • 12 GB RAM
  • Linux 4.15.0-106-generic x86_64 (Ubuntu 16.04)
  • GStreamer 1.8.3

FPS Measurements

CPU Load Measurements

Jetson AGX Xavier

The Jetson Xavier power modes used were 2 and 6 (more information: Supported Modes and Power Efficiency)

  • View current power mode:
$ sudo /usr/sbin/nvpmodel -q
  • Change current power mode:
sudo /usr/sbin/nvpmodel -m x

Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).

FPS Measurements

CPU Load Measurements

Jetson TX2

FPS Measurements

CPU Load Measurements

Jetson Nano

FPS Measurements

CPU Load Measurements

Google Coral

The following benchmarks were performed on the Coral Dev Board.

FPS Measurements

CPU Load Measurements


Previous: Example Applications/DispTec Index Next: Model Zoo



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=GstInference/Benchmarks&oldid=58370"