OpenGL Accelerated HTML Overlay/Performance/NVIDIA Jetson: Difference between revisions

From RidgeRun Developer Wiki
mNo edit summary
 
(81 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<noinclude>
<noinclude>
{{OpenGL Accelerated HTML Overlay/Head|previous=|next=|metakeywords=Contact Us}}
{{OpenGL Accelerated HTML Overlay/Head|previous=Examples/GStreamer_Usage|next=Performance/x86|metakeywords=Contact Us}}
</noinclude>
</noinclude>
= Library API performance =
The API has to major components: the hardware-accelerated graphical rendering done by OpenGL and the web rendering engine done by Webkit GTK.
In the following section you will find the performance for the API separated into this two sections.


== Graphical Rendering by OpenGL ==
__TOC__


== Library performance ==
The library has two major components: the hardware-accelerated graphical rendering done by OpenGL and the web rendering engine done by Webkit GTK.
In the following section, you will find the performance for the library separated into these two sections.
=== Graphical Rendering by OpenGL ===
<br>
In this section, we present results about the performance of HTML Overlay tested in the following setup:
* Board: NVIDIA Jetson Xavier NX
* Jetpack: 5.1
All the packages and dependencies are retrieved from the default APT repositories.
<br>
The following table shows the CPU usage, GPU usage, processing time and FPS.
The following table shows the CPU usage, GPU usage, processing time and FPS.
 
<br>
* Jetson power mode: 10W_DESKTOP
<br>
<center>
<center>
{| class="wikitable"  
{| class="wikitable" style="text-align:center;"
|-
! style="font-weight:bold; text-align:left;" | Board
! colspan="12" style="font-weight:bold;" | NVIDIA Jetson Xavier NX
|- style="font-weight:bold;"
| style="text-align:left;" | Resolution
| colspan="4" style="background-color:#9aff99;" | 4K
| colspan="4" style="background-color:#ffce93;" | +1080p
| colspan="4" style="background-color:#38fff8;" | +720p
|- style="font-weight:bold;"
| style="text-align:left;" | Measurement
| style="background-color:#9aff99;" | CPU usage (%)
| style="background-color:#9aff99;" | GPU usage (%)
| style="background-color:#9aff99;" | Processing time (ms)
| style="background-color:#9aff99;" | FPS
| style="background-color:#ffce93;" | CPU usage (%)
| style="background-color:#ffce93;" | GPU usage (%)
| style="background-color:#ffce93;" | Processing time (ms)
| style="background-color:#ffce93;" | FPS
| style="background-color:#38fff8;" | CPU usage (%)
| style="background-color:#38fff8;" | GPU usage (%)
| style="background-color:#38fff8;" | Processing time (ms)
| style="background-color:#38fff8;" | FPS
|-
|-
! rowspan="2" |  
| style="font-weight:bold; text-align:left;" | Power Configuration
! colspan="4" style="text-align:center; background-color:#ffce93;" | Jetson NX (+1080p)
| colspan="12" | 10 W Desktop Mode
|-
|-
| style="background-color:#ffd6a5;" | CPU usage (%)
| style="font-weight:bold; text-align:left;" | Upload
| style="background-color:#ffd6a5;" | GPU usage (%)
| style="background-color:#9aff99;" | 6.34
| style="background-color:#ffd6a5;" | Processing time (ms)
| style="background-color:#9aff99;" | 36.99
| style="background-color:#ffd6a5;" | FPS
| style="background-color:#9aff99;" | 11.46
| style="background-color:#9aff99;" | 87.24
| style="background-color:#ffce93;" | 7.25
| style="background-color:#ffce93;" | 19.64
| style="background-color:#ffce93;" | 14
| style="background-color:#ffce93;" | 71.4
| style="background-color:#38fff8;" | 4.44
| style="background-color:#38fff8;" | 15.72
| style="background-color:#38fff8;" | 7.293
| style="background-color:#38fff8;" | 137
|-
|-
| Upload
| style="font-weight:bold; text-align:left;" | Draw
| style="background-color:#ffd6a5;" | 1
| style="background-color:#9aff99;" | 0.16
| style="background-color:#ffd6a5;" | 2
| style="background-color:#9aff99;" | 11.96
| style="background-color:#ffd6a5;" | 3
| style="background-color:#9aff99;" | 4.329
| style="background-color:#ffd6a5;" | 4
| style="background-color:#9aff99;" | 231
| style="background-color:#ffce93;" | 0.36
| style="background-color:#ffce93;" | 7.71
| style="background-color:#ffce93;" | 2.5
| style="background-color:#ffce93;" | 395
| style="background-color:#38fff8;" | 0.40
| style="background-color:#38fff8;" | 5.67
| style="background-color:#38fff8;" | 2.077
| style="background-color:#38fff8;" | 481
|-
|-
| Draw
| style="font-weight:bold;text-align:left;" | Download
| style="background-color:#ffd6a5;" | 5
| style="background-color:#9aff99;" | 7.70
| style="background-color:#ffd6a5;" | 6
| style="background-color:#9aff99;" | 29.08
| style="background-color:#ffd6a5;" | 7
| style="background-color:#9aff99;" | 15.053
| style="background-color:#ffd6a5;" | 8
| style="background-color:#9aff99;" | 66.43
| style="background-color:#ffce93;" | 5
| style="background-color:#ffce93;" | 14.73
| style="background-color:#ffce93;" | 8.4
| style="background-color:#ffce93;" | 118
| style="background-color:#38fff8;" | 3.14
| style="background-color:#38fff8;" | 6.53
| style="background-color:#38fff8;" | 4.552
| style="background-color:#38fff8;" | 220
|-
|-
| Download
| style="font-weight:bold; text-align:left;" | Power Configuration
| style="background-color:#ffd6a5;" | 9
| colspan="12" | 20 W + Jetson Clocks (Max Power)
| style="background-color:#ffd6a5;" | 10
|- style="vertical-align:middle;"
| style="background-color:#ffd6a5;" | 11
| style="font-weight:bold; text-align:left;" | Upload
| style="background-color:#ffd6a5;" | 12
| style="background-color:#9aff99;" | 4.25
| style="background-color:#9aff99;" | 6.40
| style="background-color:#9aff99;" | 11.357
| style="background-color:#9aff99;" | 88
| style="background-color:#ffce93;" | 1.45
| style="background-color:#ffce93;" | 2.21
| style="background-color:#ffce93;" | 2.997
| style="background-color:#ffce93;" | 334
| style="background-color:#38fff8;" | 0.77
| style="background-color:#38fff8;" | 1.30
| style="background-color:#38fff8;" | 1.492
| style="background-color:#38fff8;" | 670
|- style="vertical-align:middle;"
| style="font-weight:bold; text-align:left;" | Draw
| style="background-color:#9aff99;" | 0.09
| style="background-color:#9aff99;" | 1.59
| style="background-color:#9aff99;" | 0.773
| style="background-color:#9aff99;" | 1294
| style="background-color:#ffce93;" | 0.09
| style="background-color:#ffce93;" | 0.82
| style="background-color:#ffce93;" | 0.478
| style="background-color:#ffce93;" | 2092
| style="background-color:#38fff8;" | 0.10
| style="background-color:#38fff8;" | 0.66
| style="background-color:#38fff8;" | 0.443
| style="background-color:#38fff8;" | 2309
|- style="vertical-align:middle;"
| style="font-weight:bold; text-align:left;" | Download
| style="background-color:#9aff99;" | 2.73
| style="background-color:#9aff99;" | 6.43
| style="background-color:#9aff99;" | 6.489
| style="background-color:#9aff99;" | 154.1
| style="background-color:#ffce93;" | 0.94
| style="background-color:#ffce93;" | 2.13
| style="background-color:#ffce93;" | 1.947
| style="background-color:#ffce93;" | 514
| style="background-color:#38fff8;" | 0.57
| style="background-color:#38fff8;" | 0.84
| style="background-color:#38fff8;" | 1.071
| style="background-color:#38fff8;" | 933.7
|}
|}
</center>
</center>


* Jetson power mode: 20W_6CORE, jetson clocks activated
=== Web rendering by WebkitGTK ===
The following table shows the CPU usage, GPU usage, processing time and FPS.


<br>
<center>
<center>
{| class="wikitable"
{| class="wikitable" style="text-align:center;"
|-
! rowspan="2" |
! colspan="4" style="text-align:center; background-color:#ffce93;" | Jetson NX (+1080p)
|-
|-
| style="background-color:#ffd6a5;" | CPU usage (%)
! style="font-weight:bold; text-align:left;" | Board
| style="background-color:#ffd6a5;" | GPU usage (%)
! colspan="12" style="font-weight:bold;" | NVIDIA Jetson Xavier NX
| style="background-color:#ffd6a5;" | Processing time (ms)
|- style="font-weight:bold;"
| style="background-color:#ffd6a5;" | FPS
| style="text-align:left;" | Resolution
| colspan="4" style="background-color:#9aff99;" | 4K
| colspan="4" style="background-color:#ffce93;" | +1080p
| colspan="4" style="background-color:#38fff8;" | +720p
|- style="font-weight:bold;"
| style="text-align:left;" | Measurement
| style="background-color:#9aff99;" | CPU usage (%)
| style="background-color:#9aff99;" | GPU usage (%)
| style="background-color:#9aff99;" | Processing time (ms)
| style="background-color:#9aff99;" | FPS
| style="background-color:#ffce93;" | CPU usage (%)
| style="background-color:#ffce93;" | GPU usage (%)
| style="background-color:#ffce93;" | Processing time (ms)
| style="background-color:#ffce93;" | FPS
| style="background-color:#38fff8;" | CPU usage (%)
| style="background-color:#38fff8;" | GPU usage (%)
| style="background-color:#38fff8;" | Processing time (ms)
| style="background-color:#38fff8;" | FPS
|-
|-
| Upload
| style="font-weight:bold; text-align:left;" | Power Configuration
| style="background-color:#ffd6a5;" | 1
| colspan="12" | 10 W Desktop Mode
| style="background-color:#ffd6a5;" | 2
| style="background-color:#ffd6a5;" | 3
| style="background-color:#ffd6a5;" | 4
|-
|-
| Draw
| style="font-weight:bold; text-align:left;" | Draw
| style="background-color:#ffd6a5;" | 5
| style="background-color:#9aff99;" | 9.2
| style="background-color:#ffd6a5;" | 6
| style="background-color:#9aff99;" | 0
| style="background-color:#ffd6a5;" | 7
| style="background-color:#9aff99;" | 215.425
| style="background-color:#ffd6a5;" | 8
| style="background-color:#9aff99;" | 4.64
| style="background-color:#ffce93;" | 9.2
| style="background-color:#ffce93;" | 0
| style="background-color:#ffce93;" | 52.525
| style="background-color:#ffce93;" | 19
| style="background-color:#38fff8;" | 6.77
| style="background-color:#38fff8;" | 0
| style="background-color:#38fff8;" | 23.095
| style="background-color:#38fff8;" | 43.3
|-
|-
| Download
| style="font-weight:bold; text-align:left;" | Power Configuration
| style="background-color:#ffd6a5;" | 9
| colspan="12" | 20 W + Jetson Clocks (Max Power)<ref name=performace_note>The reduced results with the 20W power mode is expected since the CPU is running at a reduced clock of 1.4GHz compared to the 1.9GHz </ref>
| style="background-color:#ffd6a5;" | 10
|- style="vertical-align:middle;"
| style="background-color:#ffd6a5;" | 11
| style="font-weight:bold; text-align:left;" | Draw
| style="background-color:#ffd6a5;" | 12
| style="background-color:#9aff99;" | 6.1
| style="background-color:#9aff99;" | 0
| style="background-color:#9aff99;" | 281.714
| style="background-color:#9aff99;" | 3.55
| style="background-color:#ffce93;" | 6.3
| style="background-color:#ffce93;" | 0
| style="background-color:#ffce93;" | 69.137
| style="background-color:#ffce93;" | 14.46
| style="background-color:#38fff8;" | 5.6
| style="background-color:#38fff8;" | 0
| style="background-color:#38fff8;" | 28.515
| style="background-color:#38fff8;" | 35.1
|- style="vertical-align:middle;"
|}
|}
</center>
</center>
* Note: When on mode 20W+Jetson Clocks the frequency of operation is 1.4 GHz and for 10w+Desktop mode the frequency of operation is 1.9 GHz. This is shown for the measurements for processing time in each mode.
* Note: There is no GPU consumption since we are using a flag that disables the use of GPU for WebkitGTK.
<source lang=bash>
export WEBKIT_DISABLE_COMPOSITING_MODE=1
</source>


= GStreamer plugin performance =
= GStreamer plugin performance =


The plugin was tested with an example overlay and a camera, using a Jetson Xavier NX with Jetpack 5.1.1 and with the 10W-4 core desktop power mode.
The plugin was tested with an example overlay and a camera, using a Jetson Xavier NX with Jetpack 5.1.1. The measurements were taken with the following pipeline, using gst-perf:
The overlay used:
<pre>
<source lang=html>
gst-launch-1.0 -ve nvarguscamerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),height=$H,width=$W,framerate=30/1" ! nvvidconv flip-method=2 ! queue ! htmloverlay url="http://0.0.0.0:8000/overlay.html" enable-js=true web-refresh-rate=10 ! perf ! queue ! nvvidconv ! xvimagesink
</pre>
<br>
 
<center>
{| class="wikitable"
|-
! Board
! colspan="12" style="text-align:center; font-weight:bold;" | Jetson Xavier NX
|- style="font-weight:bold;"
| style="font-weight:normal;" | Resolution
| style="text-align:center; background-color:#a0f8f5;" | +720p
| style="text-align:center; background-color:#f8dab4;" | +1080p
| style="text-align:center; background-color:#f8dab4;" | +4k
|-
| style="font-weight:normal;" | FPS(10W-4core)
| style="background-color:#a0f8f5;" | 166.6167
| style="background-color:#f8dab4;" | 54.7143
| style="background-color:#f8dab4;" | 13.4852
|-
| style="font-weight:normal;" | FPS(20W-6core & jetson-clocks)<ref name=performace_note/>
| style="background-color:#a0f8f5;" | 120.5662
| style="background-color:#f8dab4;" | 56.2988
| style="background-color:#f8dab4;" | 12.7323
|}
</center>
===Used overlay===
The user overlay (click [https://drive.google.com/file/d/1iflIbRJxrRg5m9s0ZhfCOWusdoCVbIF7/view?usp=sharing View Source] on the wiki to see the html):
<html>
<html>
<head>
<head>
Line 92: Line 255:
       margin-right: 3px;
       margin-right: 3px;
     }
     }
     .row {
     .row {height: 50%;}
      height: 50%;
    }
     @keyframes flickerAnimation {
     @keyframes flickerAnimation {
       0% {
       0% {opacity: 1;}
        opacity: 1;
       50% {opacity: 0;}
      }
       100% {opacity: 1;}
       50% {
        opacity: 0;
      }
       100% {
        opacity: 1;
      }
     }
     }
     @-o-keyframes flickerAnimation {
     @-o-keyframes flickerAnimation {
       0% {
       0% {opacity: 1;}
        opacity: 1;
       50% {opacity: 0;}
      }
       100% {opacity: 1;}
       50% {
        opacity: 0;
      }
       100% {
        opacity: 1;
      }
     }
     }
     @-moz-keyframes flickerAnimation {
     @-moz-keyframes flickerAnimation {
       0% {
       0% {opacity: 1;}
        opacity: 1;
       50% {opacity: 0;}
      }
       100% {opacity: 1;}
       50% {
        opacity: 0;
      }
       100% {
        opacity: 1;
      }
     }
     }
     @-webkit-keyframes flickerAnimation {
     @-webkit-keyframes flickerAnimation {
       0% {
       0% {opacity: 1;}
        opacity: 1;
       50% {opacity: 0;}
      }
       100% {opacity: 1;}
       50% {
        opacity: 0;
      }
       100% {
        opacity: 1;
      }
     }
     }
     #rec {
     #rec {
Line 210: Line 347:
</html>
</html>


</source>


[[File:Overlayfullimage.jpg|1200px|frameless|center|HTML Overlay library ]]
The following results show multiple tests for different resolutions at 30 fps, in order to dig into the multiple capabilities of the end user. You can link the limit fps of the limit tables to the average table just to realize the limits of each resolution, but remember that the limit is just virtual since we are using the element imagefreeze to set the hardware to the limit.
[[File:Overlayfullimage.jpg|1200px|frameless|center|HTML Overlay library ]]
 
[[File:Overlayfullimage.jpg|1200px|frameless|center|HTML Overlay library ]]
==Orin Nano Platform==
'''CPU usage'''
 
Taking the following pipelines as reference:
 
'''No GL Memory'''
 
For average behavior:
<pre>
gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! "video/x-raw" ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw" ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
For limit behavior:
<pre>
gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
'''Results for average behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5;" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 30
| style="text-align:center; background-color:#f8dab4;" | 30
| style="text-align:center; background-color:#a0f8f5;" | 30
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 21
| style="text-align:center; background-color:#f8dab4;" | 23
| style="text-align:center; background-color:#a0f8f5;" | 47
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 568
| style="text-align:center; background-color:#f8dab4;" | 640
| style="text-align:center; background-color:#a0f8f5;" | 1040
|}
</center>
 
'''Results for limit behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5;" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 375.123
| style="text-align:center; background-color:#f8dab4;" | 187.907
| style="text-align:center; background-color:#a0f8f5;" | 51.722
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 24
| style="text-align:center; background-color:#f8dab4;" | 24
| style="text-align:center; background-color:#a0f8f5;" | 31
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 332.564
| style="text-align:center; background-color:#f8dab4;" | 359.464
| style="text-align:center; background-color:#a0f8f5;" | 359.464
|}
</center>
 
'''GL Memory''':
 
For average behavior:
<pre>
gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
For limit behavior:
<pre>
gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
'''Results for average behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 30
| style="text-align:center; background-color:#f8dab4;" | 30
| style="text-align:center; background-color:#a0f8f5;" | 24
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 9
| style="text-align:center; background-color:#f8dab4;" | 22
| style="text-align:center; background-color:#a0f8f5;" | 32
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 424
| style="text-align:center; background-color:#f8dab4;" | 616
| style="text-align:center; background-color:#a0f8f5;" | 848
|}
</center>
 
'''Results for limit behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5;" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 793.2
| style="text-align:center; background-color:#f8dab4;" | 462.677
| style="text-align:center; background-color:#a0f8f5;" | 146.421
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 25
| style="text-align:center; background-color:#f8dab4;" | 25
| style="text-align:center; background-color:#a0f8f5;" | 36
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 301.245
| style="text-align:center; background-color:#f8dab4;" | 337.456
| style="text-align:center; background-color:#a0f8f5;" | 337.456
|}
</center>
 
==Xavier NX Platform==
'''CPU usage'''
 
Taking the following pipelines as reference:
 
'''No GL Memory'''
 
For average behavior:
<pre>
gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! "video/x-raw" ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw" ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
For limit behavior:
<pre>
gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
'''Results for average behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5;" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 30
| style="text-align:center; background-color:#f8dab4;" | 30
| style="text-align:center; background-color:#a0f8f5;" | 30
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 14
| style="text-align:center; background-color:#f8dab4;" | 17
| style="text-align:center; background-color:#a0f8f5;" | 40
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 189
| style="text-align:center; background-color:#f8dab4;" | 231
| style="text-align:center; background-color:#a0f8f5;" | 469
|}
</center>
 
'''Results for limit behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5;" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 302.5
| style="text-align:center; background-color:#f8dab4;" | 159.685
| style="text-align:center; background-color:#a0f8f5;" | 48.624
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 25
| style="text-align:center; background-color:#f8dab4;" | 29
| style="text-align:center; background-color:#a0f8f5;" | 39
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 406
| style="text-align:center; background-color:#f8dab4;" | 455
| style="text-align:center; background-color:#a0f8f5;" | 630
|}
</center>
 
'''GL Memory''':
 
For average behavior:
<pre>
gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
For limit behavior:
<pre>
gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink
</pre>
 
'''Results for average behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 30
| style="text-align:center; background-color:#f8dab4;" | 30
| style="text-align:center; background-color:#a0f8f5;" | 23
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 20
| style="text-align:center; background-color:#f8dab4;" | 26
| style="text-align:center; background-color:#a0f8f5;" | 39
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 147
| style="text-align:center; background-color:#f8dab4;" | 168
| style="text-align:center; background-color:#a0f8f5;" | 301
|}
</center>
 
'''Results for limit behavior'''
 
<center>
{| class="wikitable"
|- style="font-weight:bold;"
! Resolution
! style="text-align:center; background-color:#9aff99;" | 720p
! style="text-align:center; background-color:#f8dab4;" | 1080p<br />
! style="text-align:center; background-color:#a0f8f5;" | 4K
|-
| Max Framerate (fps)
| style="text-align:center; background-color:#9aff99;" | 450.771
| style="text-align:center; background-color:#f8dab4;" | 311.757
| style="text-align:center; background-color:#a0f8f5;" | 125.448
|-
| CPU(%)
| style="text-align:center; background-color:#9aff99;" | 27
| style="text-align:center; background-color:#f8dab4;" | 28
| style="text-align:center; background-color:#a0f8f5;" | 37
|-
| RAM(MiB)
| style="text-align:center; background-color:#9aff99;" | 378
| style="text-align:center; background-color:#f8dab4;" | 448
| style="text-align:center; background-color:#a0f8f5;" | 679
|}
</center>


<noinclude>
<noinclude>
{{OpenGL Accelerated HTML Overlay/Foot||}}
{{OpenGL Accelerated HTML Overlay/Foot|Examples/GStreamer_Usage|Performance/x86}}
</noinclude>
</noinclude>

Latest revision as of 20:31, 16 October 2023



Previous: Examples/GStreamer_Usage Index Next: Performance/x86
OpenGL Accelerated HTML Overlay
Basics
Getting Started
Library User Manual
GStreamer
Examples
Performance
Contact Us




Library performance

The library has two major components: the hardware-accelerated graphical rendering done by OpenGL and the web rendering engine done by Webkit GTK. In the following section, you will find the performance for the library separated into these two sections.

Graphical Rendering by OpenGL


In this section, we present results about the performance of HTML Overlay tested in the following setup:

  • Board: NVIDIA Jetson Xavier NX
  • Jetpack: 5.1

All the packages and dependencies are retrieved from the default APT repositories.


The following table shows the CPU usage, GPU usage, processing time and FPS.

Board NVIDIA Jetson Xavier NX
Resolution 4K +1080p +720p
Measurement CPU usage (%) GPU usage (%) Processing time (ms) FPS CPU usage (%) GPU usage (%) Processing time (ms) FPS CPU usage (%) GPU usage (%) Processing time (ms) FPS
Power Configuration 10 W Desktop Mode
Upload 6.34 36.99 11.46 87.24 7.25 19.64 14 71.4 4.44 15.72 7.293 137
Draw 0.16 11.96 4.329 231 0.36 7.71 2.5 395 0.40 5.67 2.077 481
Download 7.70 29.08 15.053 66.43 5 14.73 8.4 118 3.14 6.53 4.552 220
Power Configuration 20 W + Jetson Clocks (Max Power)
Upload 4.25 6.40 11.357 88 1.45 2.21 2.997 334 0.77 1.30 1.492 670
Draw 0.09 1.59 0.773 1294 0.09 0.82 0.478 2092 0.10 0.66 0.443 2309
Download 2.73 6.43 6.489 154.1 0.94 2.13 1.947 514 0.57 0.84 1.071 933.7

Web rendering by WebkitGTK

The following table shows the CPU usage, GPU usage, processing time and FPS.


Board NVIDIA Jetson Xavier NX
Resolution 4K +1080p +720p
Measurement CPU usage (%) GPU usage (%) Processing time (ms) FPS CPU usage (%) GPU usage (%) Processing time (ms) FPS CPU usage (%) GPU usage (%) Processing time (ms) FPS
Power Configuration 10 W Desktop Mode
Draw 9.2 0 215.425 4.64 9.2 0 52.525 19 6.77 0 23.095 43.3
Power Configuration 20 W + Jetson Clocks (Max Power)[1]
Draw 6.1 0 281.714 3.55 6.3 0 69.137 14.46 5.6 0 28.515 35.1
  • Note: When on mode 20W+Jetson Clocks the frequency of operation is 1.4 GHz and for 10w+Desktop mode the frequency of operation is 1.9 GHz. This is shown for the measurements for processing time in each mode.
  • Note: There is no GPU consumption since we are using a flag that disables the use of GPU for WebkitGTK.
export WEBKIT_DISABLE_COMPOSITING_MODE=1

GStreamer plugin performance

The plugin was tested with an example overlay and a camera, using a Jetson Xavier NX with Jetpack 5.1.1. The measurements were taken with the following pipeline, using gst-perf: 

gst-launch-1.0 -ve nvarguscamerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),height=$H,width=$W,framerate=30/1" ! nvvidconv flip-method=2 ! queue ! htmloverlay url="http://0.0.0.0:8000/overlay.html" enable-js=true web-refresh-rate=10 ! perf ! queue ! nvvidconv ! xvimagesink


Board Jetson Xavier NX
Resolution +720p +1080p +4k
FPS(10W-4core) 166.6167 54.7143 13.4852
FPS(20W-6core & jetson-clocks)[1] 120.5662 56.2988 12.7323

Used overlay

The user overlay (click View Source on the wiki to see the html):

REC
Montreal City, Canada


The following results show multiple tests for different resolutions at 30 fps, in order to dig into the multiple capabilities of the end user. You can link the limit fps of the limit tables to the average table just to realize the limits of each resolution, but remember that the limit is just virtual since we are using the element imagefreeze to set the hardware to the limit.

Orin Nano Platform

CPU usage

Taking the following pipelines as reference:

No GL Memory

For average behavior: 

gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! "video/x-raw" ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw" ! queue ! perf print-cpu-load=true ! fakesink

For limit behavior: 

gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! perf print-cpu-load=true ! fakesink

Results for average behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 30 30 30
CPU(%) 21 23 47
RAM(MiB) 568 640 1040

Results for limit behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 375.123 187.907 51.722
CPU(%) 24 24 31
RAM(MiB) 332.564 359.464 359.464

GL Memory:

For average behavior: 

gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink

For limit behavior: 

gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink

Results for average behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 30 30 24
CPU(%) 9 22 32
RAM(MiB) 424 616 848

Results for limit behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 793.2 462.677 146.421
CPU(%) 25 25 36
RAM(MiB) 301.245 337.456 337.456

Xavier NX Platform

CPU usage

Taking the following pipelines as reference:

No GL Memory

For average behavior: 

gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! "video/x-raw" ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw" ! queue ! perf print-cpu-load=true ! fakesink

For limit behavior: 

gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! htmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! perf print-cpu-load=true ! fakesink

Results for average behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 30 30 30
CPU(%) 14 17 40
RAM(MiB) 189 231 469

Results for limit behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 302.5 159.685 48.624
CPU(%) 25 29 39
RAM(MiB) 406 455 630

GL Memory:

For average behavior: 

gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,framerate=30/1,height=${H},width=${W}" ! queue ! nvvidconv ! videoconvert ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink

For limit behavior: 

gst-launch-1.0 videotestsrc num-buffers=1 pattern=ball ! "video/x-raw,format=RGBA,height=${H},width=${W}" ! imagefreeze ! queue ! glupload ! "video/x-raw(memory:GLMemory)" ! queue ! glhtmloverlay url="https://www.clocktab.com/" enable-js=true web-refresh-rate=5 overlay-x=100 ! queue ! "video/x-raw(memory:GLMemory)"  ! gldownload ! queue ! perf print-cpu-load=true ! fakesink

Results for average behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 30 30 23
CPU(%) 20 26 39
RAM(MiB) 147 168 301

Results for limit behavior

Resolution 720p 1080p
 4K
Max Framerate (fps) 450.771 311.757 125.448
CPU(%) 27 28 37
RAM(MiB) 378 448 679



Previous: Examples/GStreamer_Usage Index Next: Performance/x86



  1. ↑ 1.0 1.1 The reduced results with the 20W power mode is expected since the CPU is running at a reduced clock of 1.4GHz compared to the 1.9GHz
Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=OpenGL_Accelerated_HTML_Overlay/Performance/NVIDIA_Jetson&oldid=51059"