2,070
edits
Holoscan Framework/Holoscan Sensor Bridge/Performance: Difference between revisions
Holoscan Framework/Holoscan Sensor Bridge/Performance (view source)
Revision as of 18:22, 6 December 2024
, 6 December 2024→CUDA ISP Results
| Line 59: | Line 59: | ||
[[File:Cuda_ISP_Holoscan.png|800px|frameless|center|alt=Baseline ISP Pipeline|Baseline ISP Pipeline]] | [[File:Cuda_ISP_Holoscan.png|800px|frameless|center|alt=Baseline ISP Pipeline|Baseline ISP Pipeline]] | ||
Each of these blocks is executed in parallel for pipeline-like acceleration. Removing one of the blocks will shorten the frame processing time (latency), and further optimizing any of these blocks will also decrease the latency. | Each of these blocks is executed in parallel for pipeline-like acceleration. Removing one of the blocks will shorten the frame processing time (latency), and further optimizing any of these blocks will also decrease the latency. In this case, the CUDA ISP manages to reduce one block. | ||
On the other hand, an important consideration is that CUDA ISP does not offer RGBA64, needed for the Holoviz. This integrates the necessary conversions for RGBA64 to RGBA32 back and forth. | |||
The following table highlights the results by using the Holoscan Sensor Bridge, the NVIDIA Jetson AGX Orin and the IMX274 imager: | |||
{| class="wikitable" | |||
|- | |||
! Pipeline | |||
! Image Dimensions (px) | |||
! CPU Usage (%) | |||
! RAM Usage (MiB) | |||
! GPU Usage (%) | |||
! Glass-to-Glass Latency (ms) | |||
|- | |||
| Baseline | |||
| 3840x2160 (4K) | |||
| 2.56 (total) 30.6 (core) | |||
| 347 | |||
| 14 | |||
| 96 | |||
|- | |||
| CUDA ISP | |||
| 3840x2160 (4K) | |||
| 2.96 (total) 35.6 (core) | |||
| 401 | |||
| 35 | |||
| 58 | |||
|} | |||
This involves configuring the Jetson into a maximum performance mode. The latency lowered from 96 to 58 ms by optimizing the pipeline, leading to a 39.5% reduction. | |||
More improvement can be applied by offloading the image signal processing to the FPGA, reducing the pressure on the Jetson system. The FPGA can potentially reduce the latency given the dataflow execution pattern offered by FPGA Hardware Acceleration. RidgeRun is exploring new ways to reduce the latency to 50 ms or less for critical applications, optimizing ISP algorithms and using FPGAs. | |||
<br><br><br> | <br><br><br> | ||