Measuring CUDA ISP API performance using GstShark and GstPerf
| CUDA ISP for NVIDIA Jetson |
|---|
 |
| CUDA ISP for NVIDIA Jetson Basics |
| Getting Started |
| User Manual |
| GStreamer |
| Examples |
| Performance |
| Contact Us |
GStreamer elements performance
To measure the performance, we have used two of our GStreamer tools: GstShark and GstPerf.
For testing purposes, take into account the following points:
- Maximum performance mode enabled: all cores and Jetson clocks enabled.
- Jetson Orin Nano 8GB with JP 6.2.
- Jetson Orin Nx 16GB with JP 6.2.
- Jetson AGX Orin Devkit with JP 6.2.
- Jetson Thor T5000 with JP 7.2.
- A patch was applied to
v4l2srcto enable bayer10 captures. You can see how to apply the patch in this link: Apply patch to v4l2src
In Hardware Setup you can find more details on the setup.
The following table shows the estimated number of concurrent streams that can be processed using the three CUDA ISP elements. The estimates were calculated by dividing the measured processing framerate by the target framerate of 30 FPS per stream.
| Number of simultaneous HD@30 streams | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Platform | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||||||
| Format | I420 | RGB | I420 | RGB | I420 | RGB | I420 | RGB | ||||
| Number of streams | 25 | 27 | 14 | 18 | 9 | 12 | 9 | 13 | ||||
| Number of simultaneous 4K@30 streams | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Platform | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||||||
| Format | I420 | RGB | I420 | RGB | I420 | RGB | I420 | RGB | ||||
| Number of streams | 10 | 10 | 6 | 9 | 3 | 5 | 3 | 5 | ||||
The following table summarizes the results for each platform and resolution for I420 format.
| I420 | ||||||||
|---|---|---|---|---|---|---|---|---|
| Measurement | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||
| Resolution | HD | 4K | HD | 4K | HD | 4K | HD | 4K |
| rrcudadebayer | ||||||||
| FPS | 2164 | 1055 | 1032 | 456 | 617 | 225 | 643 | 236 |
| Processing time (ms) | 0.46 | 0.94 | 0.97 | 2.19 | 1.62 | 4.45 | 1.56 | 4.24 |
| rrcudaawb | ||||||||
| FPS | 2429 | 788 | 1325 | 580 | 813 | 279 | 841 | 296 |
| Processing time (ms) | 0.41 | 1.27 | 0.75 | 1.73 | 1.23 | 3.58 | 1.20 | 3.38 |
The following table summarizes the results for each platform and resolution for RGB format.
| RGB | ||||||||
|---|---|---|---|---|---|---|---|---|
| Measurement | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||
| Resolution | HD | 4K | HD | 4K | HD | 4K | HD | 4K |
| rrcudadebayer | ||||||||
| FPS | 2163 | 929 | 1259 | 615 | 806 | 313 | 836 | 326 |
| Processing time (ms) | 0.46 | 1.08 | 0.79 | 1.63 | 1.24 | 3.20 | 1.20 | 3.06 |
| rrcudaawb | ||||||||
| FPS | 3407 | 1134 | 2482 | 1313 | 1887 | 758 | 1863 | 807 |
| Processing time (ms) | 0.29 | 0.88 | 0.40 | 0.76 | 0.53 | 1.32 | 0.54 | 1.24 |
The following table summarizes rrcudashift results.
| rrcudashift Bayer 8 | ||||||||
|---|---|---|---|---|---|---|---|---|
| Measurement | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||
| Resolution | HD | 4K | HD | 4K | HD | 4K | HD | 4K |
| FPS | 2219 | 917 | 1739 | 962 | 1191 | 526 | 1248 | 548 |
| Processing time (ms) | 0.45 | 1.09 | 0.57 | 1.04 | 0.84 | 1.90 | 0.80 | 1.83 |
The following table shows the processing time and FPS with the 3 elements for RGB format.
| rrcudaisp RGB | ||||||||
|---|---|---|---|---|---|---|---|---|
| Measurement | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||
| Resolution | HD | 4K | HD | 4K | HD | 4K | HD | 4K |
| FPS | 833 | 328 | 568 | 292 | 383 | 156 | 394 | 163 |
| Processing time (ms) | 1.2 | 3.05 | 1.76 | 3.43 | 2.61 | 6.42 | 2.54 | 6.13 |
The following table shows the processing time and FPS with the 3 elements for I420 format.
| rrcudaisp I420 | ||||||||
|---|---|---|---|---|---|---|---|---|
| Measurement | Jetson Thor T5000 | Jetson AGX Orin | Jetson Orin NX | Jetson Orin Nano | ||||
| Resolution | HD | 4K | HD | 4K | HD | 4K | HD | 4K |
| FPS | 756 | 303 | 437 | 202 | 271 | 101 | 281 | 106 |
| Processing time (ms) | 1.32 | 3.3 | 2.29 | 4.96 | 3.69 | 9.93 | 3.56 | 9.45 |
The following graphs show the GPU usage reported by jtop for different numbers of concurrent streams. Each stream was processed using the three rrcudaisp elements.
_rrcudaisp_4K@30fps(1).png)
_rrcudaisp_HD@30fps(1).png)
Jetson Thor T5000
The processing time and FPS were measured with an input image with 4K and HD resolution from a camera sensor for all the elements.
The following pipeline measured the processing time and FPS for the rrcudashift element.
GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 videotestsrc is-live=true ! video/x-bayer,bpp=10, width=1920,height=1080,format=rggb,framerate=30/1 ! copy ! rrcudashift shift=8 ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an RGB image as output.
GST_DEBUG='GST_TRACER:7' GST_TRACERS='proctime' gst-launch-1.0 videotestsrc is-live=true ! video/x-bayer,bpp=8, width=1920,height=1080,format=rggb,framerate=30/1 ! copy ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=RGB' ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an I420 image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 videotestsrc is-live=true ! video/x-bayer,bpp=8, width=1920,height=1080,format=rggb,framerate=30/1 ! copy ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=I420' ! fakesink
The following image shows a plot with the processing time for all elements:
Jetson Orin AGX
In this case, the patch for v4l2src was modified to enable captures in bayer 12.
The processing time and FPS were measured with an input image with 2472x2064 resolution from a camera sensor for all the elements.
The following pipeline measured the processing time and FPS for the rrcudashift element.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=12, width=2472, height=2064, format=rggb' ! rrcudashift shift=5 ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an RGB image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=12, width=2472, height=2064, format=rggb' ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=RGB' ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an I420 image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=12, width=2472, height=2064, format=rggb' ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=I420' ! fakesink
The following image shows a plot with the processing time for all elements:
Jetson Orin Nano
The processing time and FPS were measured with an input image in 4K and HD resolution from a camera sensor for all the elements.
The following pipeline measured the processing time and FPS for the rrcudashift element.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, format=rggb' ! rrcudashift shift=8 ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an RGB image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=$WIDTH, height=$HEIGHT' ! rrcudadebayer ! rrcudaawb ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an I420 image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=$WIDTH, height=$HEIGHT' ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=I420' ! fakesink
The following image shows a plot with the processing time for all elements:
Jetson Orin NX
The processing time and FPS were measured with an input image with 4K and HD resolution from a camera sensor for all the elements.
The following pipeline measured the processing time and FPS for the rrcudashift element.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, format=rggb' ! rrcudashift shift=8 ! fakesink
The following pipeline was used to testing the rrcudadebayer and rrcudaawb elements with an RGB image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=$WIDTH, height=$HEIGHT' ! rrcudadebayer ! rrcudaawb ! fakesink
The following pipeline was used to test the rrcudadebayer and rrcudaawb elements with an I420 image as output.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=$WIDTH, height=$HEIGHT' ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=I420' ! fakesink
The following image shows a plot with the processing time for all elements:
More cameras
This section shows the performance results for the elements running simultaneously on multiple cameras on a Jetson Xavier AGX. For all the tests done with an RGB output image, the following pipeline was used to measure the processing time and FPS for the rrcudaawb and the rrcudadebayer element with an input image with 1920x1200 resolution coming from multiple camera sensors.
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src device=/dev/video0 io-mode=userptr ! 'video/x-bayer, bpp=10, width=1920, height=1200, format=grbg' ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=RGB' ! fakesink
In the same way, all the tests are done with an I420 output image. the following pipeline was used to measure the processing time and FPS for the rrcudaawb and the rrcudadebayer element with an input image with 1920x1200 resolution coming from multiple camera sensors
GST_DEBUG="GST_TRACER:7" GST_TRACERS="proctime" gst-launch-1.0 -ve v4l2src device=/dev/video1 io-mode=userptr ! 'video/x-bayer, bpp=10, width=1920, height=1200, format=grbg' ! rrcudadebayer ! rrcudaawb ! 'video/x-raw, format=I420' ! fakesink
The results obtained:
| rrcudadebayer | ||||||||
|---|---|---|---|---|---|---|---|---|
| Output | RGB | I420 | ||||||
| Number of cameras | Two | Three | Four | Five | Two | Three | Four | Five |
| FPS | 412 | 429 | 385 | 494 | 464 | 402 | 320 | 332 |
| Processing time (milliseconds) | 2.426 | 2.354 | 2.597 | 2.025 | 2.154 | 2.486 | 3.128 | 3.011 |
| rrcudaawb | ||||||||
|---|---|---|---|---|---|---|---|---|
| Output | RGB | I420 | ||||||
| Number of cameras | Two | Three | Four | Five | Two | Three | Four | Five |
| FPS | 397 | 374 | 689 | 347 | 429 | 450 | 289 | 296 |
| Processing time (milliseconds) | 2.521 | 2.672 | 1.450 | 2.883 | 2.330 | 2.20 | 3.459 | 3.375 |