GstInference/Benchmarks: Difference between revisions
No edit summary |
No edit summary Tag: Manual revert |
||
(84 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
<noinclude> | <noinclude> | ||
{{GstInference/Head|previous=Example Applications/DispTec|next=Model Zoo| | {{GstInference/Head|previous=Example Applications/DispTec|next=Model Zoo|metakeywords=GstInference gstreamer pipelines, Inference gstreamer pipelines, NCSDK Inference gstreamer pipelines, GoogLeNet, TinyYolo v2, GoogLeNet x2, TensorFlow backend|title=GstInference Benchmarks}} | ||
</noinclude> | </noinclude> | ||
== GstInference Benchmarks == | |||
<html> | |||
<script | |||
/* This should be added once per page only */ | |||
type="text/javascript" src="https://www.gstatic.com/charts/loader.js"> | |||
</script> | |||
</html> | |||
== GstInference Benchmarks - Introduction == | |||
This wiki summarizes a series of benchmarks on different hardware platforms based on the [https://github.com/RidgeRun/gst-inference/blob/master/tests/benchmark/run_benchmark.sh run_benchmark.sh] bash script that can be found in the official [https://github.com/RidgeRun/gst-inference GstInference repository]. The script is based on the following GStreamer pipeline: | |||
<source lang="bash"> | <source lang="bash"> | ||
#Script to run each model | |||
run_all_models(){ | |||
model_array=(inceptionv1 inceptionv2 inceptionv3 inceptionv4 tinyyolov2 tinyyolov3) | |||
</source> | model_upper_array=(InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3) | ||
The | input_array=(input input input input input/Placeholder inputs ) | ||
output_array=(InceptionV1/Logits/Predictions/Reshape_1 Softmax InceptionV3/Predictions/Reshape_1 | |||
InceptionV4/Logits/Predictions add_8 output_boxes ) | |||
mkdir -p logs/ | |||
rm -f logs/* | |||
for ((i=0;i<${#model_array[@]};++i)); do | |||
echo Perf ${model_array[i]} | |||
gst-launch-1.0 \ | |||
filesrc location=$VIDEO_PATH num-buffers=600 ! decodebin ! videoconvert ! \ | |||
perf print-arm-load=true name=inputperf ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue ! net.sink_bypass \ | |||
${model_array[i]} backend=$BACKEND name=net backend::input-layer=${input_array[i]} backend::output-layer=${output_array[i]} \ | |||
model-location="${MODELS_PATH}${model_upper_array[i]}_${INTERNAL_PATH}/graph_${model_array[i]}${EXTENSION}" \ | |||
net.src_bypass ! perf print-arm-load=true name=outputperf ! videoconvert ! fakesink sync=false > logs/${model_array[i]}.log | |||
done | |||
} | |||
</source> | |||
=== Test benchmark video === | |||
The following video was used to perform the benchmark tests. | |||
<br> | |||
To download the video press right-click on the video and select 'Save video as' and save this on your computer. | |||
<br> | |||
<br> | |||
[[File:Test benchmark video.mp4|thumb|border|center|500px|alt=Alt|Video 1. Test benchmark video]] | |||
== x86 == | |||
The Desktop PC had the following specifications: | The Desktop PC had the following specifications: | ||
*Intel(R) Core(TM) i7- | *Intel(R) Core(TM) Core i7-7700HQ CPU @ 2.80GHz | ||
* | *12 GB RAM | ||
*Linux 4.15.0-106-generic x86_64 (Ubuntu 16.04) | |||
*Linux 4.15.0- | *GStreamer 1.8.3 | ||
=== FPS Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_fps_x86" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffx86Fps); | |||
function drawStuffx86Fps() { | |||
var chartDiv_Fps_x86 = document.getElementById('chart_fps_x86'); | |||
var table_models_fps_x86 = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'ONNXRT \n x86', | |||
'ONNXRT OpenVINO (CPU_FP32) \n x86', | |||
'ONNXRT OpenVINO (GPU_FP32)\n x86', | |||
'ONNXRT OpenVINO (GPU_FP16)\n x86', | |||
'ONNXRT OpenVINO (MYRIAD_FP16)\n x86', | |||
'TensorFlow \n x86', | |||
'TensorFlow Lite \n x86'], | |||
['InceptionV1', 47.9, 81.966, 70.580, 98.742, 46.294, 55.3182, 18.8422], //row 1 | |||
['InceptionV2', 32.7, 63.352, 54.159, 77.449, 34.613, 39.6438, 13.5714], //row 2 | |||
['InceptionV3', 12.1, 23.287, 20.878, 34.059, 11.999, 16.2488, 4.9924], //row 3 | |||
['InceptionV4', 5.26, 10.927, 6.160, 4.548, 6.494, 7.793, 2.583], //row 4 | |||
['TinyYoloV2', 33.559, 32.587, 0, 0, 0, 18.1846, 7.2708], //row 5 | |||
['TinyYoloV3', 35.092, 27.799, 0, 0, 0, 21.7334, 7.3042] //row 6 | |||
]); | |||
var x86_materialOptions_fps = { | |||
width: 1000, | |||
chart: { | |||
title: 'Model Vs FPS per backend', | |||
}, | |||
series: { | |||
}, | |||
axes: { | |||
y: { | |||
distance: {side: 'left',label: 'FPS'}, // Left y-axis. | |||
} | |||
} | |||
}; | |||
var materialChart_x86_fps = new google.charts.Bar(chartDiv_Fps_x86); | |||
view_x86_fps = new google.visualization.DataView(table_models_fps_x86); | |||
function drawMaterialChart() { | |||
var materialChart_x86_fps = new google.charts.Bar(chartDiv_Fps_x86); | |||
materialChart_x86_fps.draw(table_models_fps_x86, google.charts.Bar.convertOptions(x86_materialOptions_fps)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_x86_fps.setColumns([0,1, 2, 3, 4, 5, 6, 7]); | |||
materialChart_x86_fps.draw(view_x86_fps, x86_materialOptions_fps); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
=== CPU Load Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_cpu_x86" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffx86Cpu); | |||
function drawStuffx86Cpu() { | |||
var chartDiv_Cpu_x86 = document.getElementById('chart_cpu_x86'); | |||
var table_models_cpu_x86 = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'ONNXRT \n x86', | |||
'ONNXRT OpenVINO (CPU_FP32) \n x86', | |||
'ONNXRT OpenVINO (GPU_FP32)\n x86', | |||
'ONNXRT OpenVINO (GPU_FP16)\n x86', | |||
'ONNXRT OpenVINO (MYRIAD_FP16)\n x86', | |||
'TensorFlow \n x86', | |||
'TensorFlow Lite \n x86'], //Column 1 | |||
['InceptionV1', 94.6, 49, 31, 29, 14, 74.2, 47.6], //row 1 | |||
['InceptionV2', 100, 52, 28, 29, 11, 74.2, 43.6], //row 2 | |||
['InceptionV3', 95.2, 49, 28, 28, 13, 81, 60.2], //row 3 | |||
['InceptionV4', 88.8, 49, 33, 46, 11, 86, 50], //row 4 | |||
['TinyYoloV2', 94, 50, 0, 0, 0, 80.6, 46], //row 5 | |||
['TinyYoloV3', 91.4, 46, 0, 0, 0, 74.6, 42.4] //row 6 | |||
]); | |||
var x86_materialOptions_cpu = { | |||
width: 1000, | |||
chart: { | |||
title: 'Model Vs CPU Load per backend', | |||
}, | |||
series: { | |||
}, | |||
axes: { | |||
y: { | |||
distance: {side: 'left',label: 'CPU Load'}, // Left y-axis. | |||
} | |||
} | |||
}; | |||
var materialChart_x86_cpu = new google.charts.Bar(chartDiv_Cpu_x86); | |||
view_x86_cpu = new google.visualization.DataView(table_models_cpu_x86); | |||
function drawMaterialChart() { | |||
var materialChart_x86_cpu = new google.charts.Bar(chartDiv_Cpu_x86); | |||
materialChart_x86_cpu.draw(table_models_cpu_x86, google.charts.Bar.convertOptions(x86_materialOptions_cpu)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_x86_cpu.setColumns([0,1, 2, 3, 4, 5, 6, 7]); | |||
materialChart_x86_cpu.draw(view_x86_cpu, x86_materialOptions_cpu); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
== Jetson AGX Xavier == | |||
The Jetson Xavier power modes used were 2 and 6 (more information: [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fpower_management_jetson_xavier.html%23wwpID0E0OM0HA Supported Modes and Power Efficiency]) | The Jetson Xavier power modes used were 2 and 6 (more information: [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fpower_management_jetson_xavier.html%23wwpID0E0OM0HA Supported Modes and Power Efficiency]) | ||
Line 39: | Line 231: | ||
Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6). | Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6). | ||
=== | === FPS Measurements === | ||
<html> | |||
<style> | |||
.button { | .button { | ||
background-color: #008CBA; | background-color: #008CBA; | ||
Line 164: | Line 249: | ||
} | } | ||
</style> | </style> | ||
<div id="chart_fps_xavier" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<div id=" | |||
<script> | <script> | ||
google.charts.load('current', {'packages':['corechart', 'bar']}); | google.charts.load('current', {'packages':['corechart', 'bar']}); | ||
google.charts.setOnLoadCallback( | google.charts.setOnLoadCallback(drawStuffXavierFps); | ||
function drawStuffXavierFps() { | |||
var chartDiv_Fps_Xavier = document.getElementById('chart_fps_xavier'); | |||
var | |||
var | var table_models_fps_xavier = google.visualization.arrayToDataTable([ | ||
[' | ['Model', //Column 0 | ||
' | 'TensorFlow \n Xavier (15 W)', //Column 1 | ||
' | 'TensorFlow (GPU) \n Xavier (15 W)', //Column 2 | ||
' | 'TensorFlow \n Xavier (30 W)', //Column 3 | ||
' | 'TensorFlow (GPU) \n Xavier (30 W)', //Column 4 | ||
' | 'TensorRT \n Xavier', //Column 5 | ||
' | 'ONNXRT ACL \n Xavier'], //Column 6 | ||
['InceptionV1', 8.24, 52.3, 6.41, 66.27, 92.6, 17.566], //row 1 | |||
['InceptionV2', 6.58, 39.6, 5.11, 50.59, 0, 12.729], //row 2 | |||
['InceptionV3', 2.54, 17.8, 1.96, 22.95, 24.9, 5.709], //row 3 | |||
['InceptionV4', 1.22, 9.4, 0.98, 12.14, 13.6, 2.747], //row 4 | |||
['TinyYoloV2', 0, 0, 0, 0, 69.7, 9.367], //row 5 | |||
['TinyYoloV3', 0, 0, 0, 0, 0, 10.520] //row 6 | |||
[' | |||
['InceptionV2', | |||
['InceptionV3', | |||
['InceptionV4', | |||
['TinyYoloV2', | |||
['TinyYoloV3', | |||
]); | ]); | ||
var | var xavier_materialOptions_fps = { | ||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: 'Model Vs | title: 'Model Vs FPS per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 290: | Line 288: | ||
} | } | ||
}; | }; | ||
var | |||
var materialChart_xavier_fps = new google.charts.Bar(chartDiv_Fps_Xavier); | |||
view_xavier_fps = new google.visualization.DataView(table_models_fps_xavier); | |||
function drawMaterialChart() { | |||
var materialChart_xavier_fps = new google.charts.Bar(chartDiv_Fps_Xavier); | |||
materialChart_xavier_fps.draw(table_models_fps_xavier, google.charts.Bar.convertOptions(xavier_materialOptions_fps)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_xavier_fps.setColumns([0,1,2,3,4,5,6]); | |||
materialChart_xavier_fps.draw(view_xavier_fps, xavier_materialOptions_fps); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
=== CPU Load Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_cpu_xavier" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffXavierCpu); | |||
function drawStuffXavierCpu() { | |||
var chartDiv_Cpu_Xavier = document.getElementById('chart_cpu_xavier'); | |||
var table_models_cpu_xavier = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'TensorFlow \n Xavier (15 W)', //Column 1 | |||
'TensorFlow (GPU) \n Xavier (15 W)', //Column 2 | |||
'TensorFlow \n Xavier (30 W)', //Column 3 | |||
'TensorFlow (GPU) \n Xavier (30 W)', //Column 4 | |||
'TensorRT \n Xavier', //Column 5 | |||
'ONNXRT ACL \n Xavier'], //Column 6 | |||
['InceptionV1', 86, 72, 93, 72, 32, 50], //row 1 | |||
['InceptionV2', 88, 62.6, 95, 62, 0, 49], //row 2 | |||
['InceptionV3', 92, 44, 98, 44, 6, 50], //row 3 | |||
['InceptionV4', 94, 32, 99, 32, 3, 50], //row 4 | |||
['TinyYoloV2', 0, 0, 0, 0, 16, 50], //row 5 | |||
['TinyYoloV3', 0, 0, 0, 0, 0, 50] //row 6 | |||
]); | |||
var xavier_materialOptions_cpu = { | |||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: 'Model Vs | title: 'Model Vs CPU load per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 299: | Line 362: | ||
axes: { | axes: { | ||
y: { | y: { | ||
distance: {side: 'left',label: ' | distance: {side: 'left',label: 'CPU Load'}, // Left y-axis. | ||
} | } | ||
} | } | ||
}; | }; | ||
var | |||
var materialChart_xavier_cpu = new google.charts.Bar(chartDiv_Cpu_Xavier); | |||
view_xavier_cpu = new google.visualization.DataView(table_models_cpu_xavier); | |||
function drawMaterialChart() { | |||
var materialChart_xavier_cpu = new google.charts.Bar(chartDiv_Cpu_Xavier); | |||
materialChart_xavier_cpu.draw(table_models_cpu_xavier, google.charts.Bar.convertOptions(xavier_materialOptions_cpu)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_xavier_cpu.setColumns([0,1,2,3,4,5,6]); | |||
materialChart_xavier_cpu.draw(view_xavier_cpu, xavier_materialOptions_cpu); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
== Jetson TX2 == | |||
=== FPS Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_fps_tx2" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffTx2Fps); | |||
function drawStuffTx2Fps() { | |||
var chartDiv_Fps_Tx2 = document.getElementById('chart_fps_tx2'); | |||
var table_models_fps_tx2 = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'TensorRT \n TX2'], //Column 1 | |||
['InceptionV1', 62.5], //row 1 | |||
['InceptionV2', 0], //row 2 | |||
['InceptionV3', 20.2], //row 3 | |||
['InceptionV4', 11.22], //row 4 | |||
['TinyYoloV2', 33.8], //row 5 | |||
['TinyYoloV3', 0] //row 6 | |||
]); | |||
var Tx2_materialOptions_fps = { | |||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: 'Model Vs | title: 'Model Vs FPS per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 317: | Line 442: | ||
}; | }; | ||
var | var materialChart_tx2_fps = new google.charts.Bar(chartDiv_Fps_Tx2); | ||
view_tx2_fps = new google.visualization.DataView(table_models_fps_tx2); | |||
function drawMaterialChart() { | function drawMaterialChart() { | ||
var | var materialChart_tx2_fps = new google.charts.Bar(chartDiv_Fps_Tx2); | ||
materialChart_tx2_fps.draw(table_models_fps_tx2, google.charts.Bar.convertOptions(Tx2_materialOptions_fps)); | |||
init_charts(); | init_charts(); | ||
} | } | ||
function init_charts(){ | function init_charts(){ | ||
view_tx2_fps.setColumns([0,1]); | |||
materialChart_tx2_fps.draw(view_tx2_fps, Tx2_materialOptions_fps); | |||
} | } | ||
drawMaterialChart(); | |||
} | } | ||
</script> | </script> | ||
</html> | </html> | ||
=== CPU | === CPU Load Measurements === | ||
<html> | <html> | ||
< | <style> | ||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_cpu_tx2" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<div id=" | |||
<script> | <script> | ||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback( | google.charts.setOnLoadCallback(drawStuffTx2Cpu); | ||
function drawStuffTx2Cpu() { | |||
var chartDiv_Cpu_Tx2 = document.getElementById('chart_cpu_tx2'); | |||
var | |||
var | var table_models_cpu_tx2 = google.visualization.arrayToDataTable([ | ||
[' | ['Model', //Column 0 | ||
' | 'TensorRT \n TX2'], //Column 1 | ||
['InceptionV1', 21], //row 1 | |||
['InceptionV2', 0], //row 2 | |||
['InceptionV3', 10], //row 3 | |||
['InceptionV4', 7], //row 4 | |||
['TinyYoloV2', 19], //row 5 | |||
['TinyYoloV3', 0] //row 6 | |||
[' | |||
[' | |||
[' | |||
[' | |||
[' | |||
[' | |||
]); | ]); | ||
var | var Tx2_materialOptions_cpu = { | ||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: ' | title: 'Model Vs CPU Load per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 573: | Line 510: | ||
axes: { | axes: { | ||
y: { | y: { | ||
distance: {side: 'left',label: ' | distance: {side: 'left',label: 'FPS'}, // Left y-axis. | ||
} | } | ||
} | } | ||
}; | }; | ||
var | |||
var materialChart_tx2_cpu = new google.charts.Bar(chartDiv_Cpu_Tx2); | |||
view_tx2_cpu = new google.visualization.DataView(table_models_cpu_tx2); | |||
function drawMaterialChart() { | |||
var materialChart_tx2_cpu = new google.charts.Bar(chartDiv_Cpu_Tx2); | |||
materialChart_tx2_cpu.draw(table_models_cpu_tx2, google.charts.Bar.convertOptions(Tx2_materialOptions_cpu)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_tx2_cpu.setColumns([0,1]); | |||
materialChart_tx2_cpu.draw(view_tx2_cpu, Tx2_materialOptions_cpu); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
== Jetson Nano == | |||
=== FPS Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_fps_nano" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffNanoFps); | |||
function drawStuffNanoFps() { | |||
var chartDiv_Fps_Nano = document.getElementById('chart_fps_nano'); | |||
var table_models_fps_nano = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'TensorRT \n Nano'], //Column 1 | |||
['InceptionV1', 36.8], //row 1 | |||
['InceptionV2', 0], //row 2 | |||
['InceptionV3', 10.9], //row 3 | |||
['InceptionV4', 4.82], //row 4 | |||
['TinyYoloV2', 14.6], //row 5 | |||
['TinyYoloV3', 0] //row 6 | |||
]); | |||
var Nano_materialOptions_fps = { | |||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: 'Model Vs | title: 'Model Vs FPS per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 586: | Line 585: | ||
axes: { | axes: { | ||
y: { | y: { | ||
distance: {side: 'left',label: ' | distance: {side: 'left',label: 'FPS'}, // Left y-axis. | ||
} | } | ||
} | } | ||
}; | }; | ||
var | |||
var materialChart_nano_fps = new google.charts.Bar(chartDiv_Fps_Nano); | |||
view_nano_fps = new google.visualization.DataView(table_models_fps_nano); | |||
function drawMaterialChart() { | |||
var materialChart_nano_fps = new google.charts.Bar(chartDiv_Fps_Nano); | |||
materialChart_nano_fps.draw(table_models_fps_nano, google.charts.Bar.convertOptions(Nano_materialOptions_fps)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_nano_fps.setColumns([0,1]); | |||
materialChart_nano_fps.draw(view_nano_fps, Nano_materialOptions_fps); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
=== CPU Load Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_cpu_nano" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffNanoCpu); | |||
function drawStuffNanoCpu() { | |||
var chartDiv_Cpu_Nano = document.getElementById('chart_cpu_nano'); | |||
var table_models_cpu_nano = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'TensorRT \n Nano'], //Column 1 | |||
['InceptionV1', 18], //row 1 | |||
['InceptionV2', 0], //row 2 | |||
['InceptionV3', 18], //row 3 | |||
['InceptionV4', 20], //row 4 | |||
['TinyYoloV2', 20], //row 5 | |||
['TinyYoloV3', 0] //row 6 | |||
]); | |||
var Nano_materialOptions_cpu = { | |||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: 'Model Vs | title: 'Model Vs CPU Load per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 599: | Line 658: | ||
axes: { | axes: { | ||
y: { | y: { | ||
distance: {side: 'left',label: ' | distance: {side: 'left',label: 'FPS'}, // Left y-axis. | ||
} | } | ||
} | } | ||
}; | }; | ||
var | var materialChart_nano_cpu = new google.charts.Bar(chartDiv_Cpu_Nano); | ||
view_nano_cpu = new google.visualization.DataView(table_models_cpu_nano); | |||
function drawMaterialChart() { | function drawMaterialChart() { | ||
var | var materialChart_nano_cpu = new google.charts.Bar(chartDiv_Cpu_Nano); | ||
materialChart_nano_cpu.draw(table_models_cpu_nano, google.charts.Bar.convertOptions(Nano_materialOptions_cpu)); | |||
init_charts(); | init_charts(); | ||
} | } | ||
function init_charts(){ | function init_charts(){ | ||
view_nano_cpu.setColumns([0,1]); | |||
materialChart_nano_cpu.draw(view_nano_cpu, Nano_materialOptions_cpu); | |||
} | } | ||
drawMaterialChart(); | |||
} | } | ||
</script> | </script> | ||
</html> | </html> | ||
=== | == Google Coral == | ||
The following benchmarks were performed on the Coral Dev Board. | |||
=== FPS Measurements === | === FPS Measurements === | ||
<html> | <html> | ||
<style> | <style> | ||
Line 784: | Line 706: | ||
</style> | </style> | ||
<div id=" | <div id="chart_fps_coral" style="margin: auto; width: 800px; height: 500px;"></div> | ||
<script> | <script> | ||
google.charts.load('current', {'packages':['corechart', 'bar']}); | google.charts.load('current', {'packages':['corechart', 'bar']}); | ||
google.charts.setOnLoadCallback( | google.charts.setOnLoadCallback(drawStuffCoralFps); | ||
function | function drawStuffCoralFps() { | ||
var | var chartDiv_Fps_Coral = document.getElementById('chart_fps_coral'); | ||
var | var table_models_fps_coral = google.visualization.arrayToDataTable([ | ||
[' | ['Model', //Column 0 | ||
' | 'TensorFlow Lite \n Coral', | ||
' | 'TensorFlow Lite EdgeTPU \n Coral'], //Column 1 | ||
['InceptionV1', 3.11, 41.6], //row 1 | |||
['InceptionV2', 2.31, 42.8], //row 2 | |||
['InceptionV3', 0.9, 15.02], //row 3 | |||
['InceptionV4', 0, 8.56], //row 4 | |||
[' | ['MobileNetV2', 0, 41.12], //row 5 | ||
['MobileNetV2 + SSD', 0, 38.64] //row 6 | |||
]); | ]); | ||
var | var Coral_materialOptions_fps = { | ||
width: 900, | |||
width: | |||
chart: { | chart: { | ||
title: ' | title: 'Model Vs FPS per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 843: | Line 740: | ||
} | } | ||
}; | }; | ||
var | |||
var materialChart_coral_fps = new google.charts.Bar(chartDiv_Fps_Coral); | |||
view_coral_fps = new google.visualization.DataView(table_models_fps_coral); | |||
function drawMaterialChart() { | |||
var materialChart_coral_fps = new google.charts.Bar(chartDiv_Fps_Coral); | |||
materialChart_coral_fps.draw(table_models_fps_coral, google.charts.Bar.convertOptions(Coral_materialOptions_fps)); | |||
init_charts(); | |||
} | |||
function init_charts(){ | |||
view_coral_fps.setColumns([0,1, 2]); | |||
materialChart_coral_fps.draw(view_coral_fps, Coral_materialOptions_fps); | |||
} | |||
drawMaterialChart(); | |||
} | |||
</script> | |||
</html> | |||
=== CPU Load Measurements === | |||
<html> | |||
<style> | |||
.button { | |||
background-color: #008CBA; | |||
border: none; | |||
color: white; | |||
padding: 15px 32px; | |||
text-align: center; | |||
text-decoration: none; | |||
display: inline-block; | |||
font-size: 16px; | |||
margin: 4px 2px; | |||
cursor: pointer; | |||
} | |||
</style> | |||
<div id="chart_cpu_coral" style="margin: auto; width: 800px; height: 500px;"></div> | |||
<script> | |||
google.charts.load('current', {'packages':['corechart', 'bar']}); | |||
google.charts.setOnLoadCallback(drawStuffCoralCpu); | |||
function drawStuffCoralCpu() { | |||
var chartDiv_Cpu_Coral = document.getElementById('chart_cpu_coral'); | |||
var table_models_cpu_coral = google.visualization.arrayToDataTable([ | |||
['Model', //Column 0 | |||
'TensorFlow Lite \n Coral', | |||
'TensorFlow Lite EdgeTPU \n Coral'], //Column 1 | |||
['InceptionV1', 73, 32], //row 1 | |||
['InceptionV2', 72, 37], //row 2 | |||
['InceptionV3', 74, 14], //row 3 | |||
['InceptionV4', 0, 5], //row 4 | |||
['MobileNetV2', 0, 34], //row 5 | |||
['MobileNetV2 + SSD', 0, 45] //row 6 | |||
]); | |||
var Coral_materialOptions_cpu = { | |||
width: 900, | width: 900, | ||
chart: { | chart: { | ||
title: 'Model Vs | title: 'Model Vs CPU Load per backend', | ||
}, | }, | ||
series: { | series: { | ||
Line 852: | Line 810: | ||
axes: { | axes: { | ||
y: { | y: { | ||
distance: {side: 'left',label: ' | distance: {side: 'left',label: 'CPU Load'}, // Left y-axis. | ||
} | } | ||
} | } | ||
}; | }; | ||
var | var materialChart_coral_cpu = new google.charts.Bar(chartDiv_Cpu_Coral); | ||
view_coral_cpu = new google.visualization.DataView(table_models_cpu_coral); | |||
function drawMaterialChart() { | function drawMaterialChart() { | ||
var | var materialChart_coral_cpu = new google.charts.Bar(chartDiv_Cpu_Coral); | ||
materialChart_coral_cpu.draw(table_models_cpu_coral, google.charts.Bar.convertOptions(Coral_materialOptions_cpu)); | |||
init_charts(); | init_charts(); | ||
} | } | ||
function init_charts(){ | function init_charts(){ | ||
view_coral_cpu.setColumns([0,1, 2]); | |||
materialChart_coral_cpu.draw(view_coral_cpu, Coral_materialOptions_cpu); | |||
} | |||
drawMaterialChart(); | |||
} | } | ||
</script> | </script> | ||
</html> | </html> | ||
<noinclude> | <noinclude> | ||
{{GstInference/Foot|Example Applications/DispTec|Model Zoo}} | {{GstInference/Foot|Example Applications/DispTec|Model Zoo}} | ||
</noinclude> | </noinclude> |
Latest revision as of 20:39, 4 September 2024
Make sure you also check GstInference's companion project: R2Inference |
GstInference |
---|
Introduction |
Getting started |
Supported architectures |
InceptionV1 InceptionV3 YoloV2 AlexNet |
Supported backends |
Caffe |
Metadata and Signals |
Overlay Elements |
Utils Elements |
Legacy pipelines |
Example pipelines |
Example applications |
Benchmarks |
Model Zoo |
Project Status |
Contact Us |
|
GstInference Benchmarks - Introduction
This wiki summarizes a series of benchmarks on different hardware platforms based on the run_benchmark.sh bash script that can be found in the official GstInference repository. The script is based on the following GStreamer pipeline:
#Script to run each model run_all_models(){ model_array=(inceptionv1 inceptionv2 inceptionv3 inceptionv4 tinyyolov2 tinyyolov3) model_upper_array=(InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3) input_array=(input input input input input/Placeholder inputs ) output_array=(InceptionV1/Logits/Predictions/Reshape_1 Softmax InceptionV3/Predictions/Reshape_1 InceptionV4/Logits/Predictions add_8 output_boxes ) mkdir -p logs/ rm -f logs/* for ((i=0;i<${#model_array[@]};++i)); do echo Perf ${model_array[i]} gst-launch-1.0 \ filesrc location=$VIDEO_PATH num-buffers=600 ! decodebin ! videoconvert ! \ perf print-arm-load=true name=inputperf ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue ! net.sink_bypass \ ${model_array[i]} backend=$BACKEND name=net backend::input-layer=${input_array[i]} backend::output-layer=${output_array[i]} \ model-location="${MODELS_PATH}${model_upper_array[i]}_${INTERNAL_PATH}/graph_${model_array[i]}${EXTENSION}" \ net.src_bypass ! perf print-arm-load=true name=outputperf ! videoconvert ! fakesink sync=false > logs/${model_array[i]}.log done }
Test benchmark video
The following video was used to perform the benchmark tests.
To download the video press right-click on the video and select 'Save video as' and save this on your computer.
x86
The Desktop PC had the following specifications:
- Intel(R) Core(TM) Core i7-7700HQ CPU @ 2.80GHz
- 12 GB RAM
- Linux 4.15.0-106-generic x86_64 (Ubuntu 16.04)
- GStreamer 1.8.3
FPS Measurements
CPU Load Measurements
Jetson AGX Xavier
The Jetson Xavier power modes used were 2 and 6 (more information: Supported Modes and Power Efficiency)
- View current power mode:
$ sudo /usr/sbin/nvpmodel -q
- Change current power mode:
sudo /usr/sbin/nvpmodel -m x
Where x is the power mode ID (e.g. 0, 1, 2, 3, 4, 5, 6).
FPS Measurements
CPU Load Measurements
Jetson TX2
FPS Measurements
CPU Load Measurements
Jetson Nano
FPS Measurements
CPU Load Measurements
Google Coral
The following benchmarks were performed on the Coral Dev Board.
FPS Measurements
CPU Load Measurements