GStreamer Based Image Signal Processor - Performance - imx8
| GStreamer Image Signal Processor |
|---|
 |
| Image Signal Processor Basics |
| Docs |
| Getting Started |
| Examples |
| Performance |
| Troubleshoot |
| Contact Us |
Benchmark environment
The following setup was used for all the benchmarks:
- NXP® i.MX 8MQuad Evaluation Kit (EVK)
- Yocto built image (fsl-image-multimedia) from Freescale BSP
- i.MX8 5640 RAW10 Camera Module
GStreamer Bayer10 Support
GStreamer has support for 8-bit Bayer only. To enable 10-bit Bayer support, please, follow this wiki.
Performance in other i.MX 8 boards
According to the product comparison available here:
| Family | Subfamilies | GPU | Number of Shaders | Supported by GstISP |
|---|---|---|---|---|
| i.MX 8 | QM, QP |
2x GC7000XSVX | 8 shader/each | Yes |
| i.MX 8X | QXP, DXP, DX | 1x GC7000Lite | 4 shaders | Yes |
| i.MX 8M | Q, D, QL |
1x GC7000Lite | 4 shaders | Yes |
| i.MX 8M Mini | QL, DL, SL, Q, D, S | 1x GCNanoUltra | 1 shader | No (No OpenCL supported) |
| i.MX 8M Nano | QL, DL, SL, Q, D, S | (Q, D, S) 1x GC7000UltraLite (QL, DL, SL) No graphics |
2 shaders | Yes in Q, D, S |
| i.MX 8M Plus | Q, QL, D | 1x GC7000UltraLite | 2 shaders | Yes |
For the measurements, we are using the i.MX 8M Quad (Q), equipped with a GPU GC7000 Lite with 4 shaders. GPUs with fewer shaders may lead to lower performance. We have observed that the ispcldebayer (Debayer) at 1080p works at ~22 fps in an i.MX 8MP is equipped with a GPU with only 2 shaders.
If you are interested in optimisation and achieving higher performance, you can message us at support@ridgerun.com, and we will be glad to team up with you on your project.
CSC Benchmarking
The following section describes some relevant performance measurements for the RGBA to NV12 color space conversion for a few standard resolutions. It begins with a summary of the results detailed in the following sections, as well as the pipelines used to capture such results.
Pipeline
All the measurements below were made using the following minimal pipeline.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw,format=(string)RGBA,width=$WIDTH,height=$HEIGHT,framerate=(fraction)30/1" ! queue ! ispclcsc ! fakesink -v
Summary
In this summary you can find measurements related with CPU usage, memory, execution time and maximum framerate supported by the CSC algorithm.
| Resolution | CPU usage (%) | Memory Bandwidth (GHz) | Execution Time (ms) |
|---|---|---|---|
| 640x480@30fps | 4 | 1.020 | 2.24 |
| 1280x720@30fps | 10 | 1.034 | 6.19 |
| 1920x1080@30fps | 14 | 1.466 | 13.17 |
| Resolution | Maximum framerate (fps) * |
|---|---|
| 640x480 | 333 |
| 1280x720 | 120 |
| 1920x1080 | 55 |
 | Note: take into consideration that these values were taken using the videotestsrc element, and may decrease with a camera source capture. |
CPU usage
These measurements were taken with the top app to monitor the behavior of the pipeline while running.
Memory usage
These measurements were taken with perf stat Linux tool to measure the bandwidth needed by the pipeline.
Execution time
These measurements were taken using some timestamps before and after the processing to measure just the execution time for the kernel algorithm on each frame and hence avoiding possible false measurements from the GStreamer plugin.
Maximum Framerate
These measurements were taken with the gst-perf tool to monitor the behavior of the pipeline while running. When using videotestsrc we are able to process as many buffers as the element can handle, hence the maximum value reported.
The following pipeline was used to test the maximum framerate, please note the use of videotestsrc.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 videotestsrc ! "video/x-raw,format=(string)RGBA,width=$WIDTH,height=$HEIGHT,framerate=(fraction)30/1" ! queue ! ispclcsc ! fakesink -v
Debayer Benckmarking
The following section describes some relevant performance measurements during debayering for a few standard resolutions. It begins with a summary of the results detailed on the following sections, as well as the pipelines used to capture such results.
Pipeline
All the measurements below were made using the following minimal pipeline.
WIDTH=1920 HEIGHT=1080 $ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-bayer,format=bggr,bpp=10,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! ispclbayer10to8 ! queue ! ispcldebayer ! fakesink -v
Summary
In this summary you can find measurements related with CPU usage, memory, execution time and maximum framerate supported by the debayer algorithm.
| Resolution | CPU usage (%) | Memory Bandwidth (GHz) | Execution Time (ms) |
|---|---|---|---|
| 640x480@30fps | 2 | 1.025 | 3.54 |
| 1280x720@30fps | 2 | 1.084 | 9.89 |
| 1920x1080@30fps | 3 | 1.402 | 21.46 |
| Resolution | Maximum framerate (fps)* |
|---|---|
| 640x480 | 266 |
| 1280x720 | 101 |
| 1920x1080 | 46 |
| Note: take into consideration that these values were taken using the videotestsrc element, and may decrease with a camera source capture. |
CPU usage
These measurements were taken with the top app to monitor the behavior of the pipeline while running.
Memory usage
These measurements were taken with perf stat linux tool to measure the bandwidth needed by the pipeline.
Execution time
These measurements were taken using some timestamps before and after the processing to measure just the execution time for the kernel algorithm on each frame and hence avoiding possible false measurements from the GStreamer plugin.
Maximum Framerate
These measurements were taken with the gst-perf tool to monitor the behavior of the pipeline while running. When using videotestsrc we are able to process as many buffers as the element can handle, hence the maximum value reported.
The following pipeline was used to test the maximum framerate, please note the use of videotestsrc.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 videotestsrc ! "video/x-bayer,format=bggr,width=$WIDTH,height=$HEIGHT,framerate=30/1" ! queue ! ispcldebayer ! perf ! fakesink -v
AWB Benchmarking
The following section describes some relevant performance measurements for the auto-white balance for a few standard resolutions. It begins with a summary of the results detailed on the following sections, as well as the pipelines used to capture such results.
Pipeline
All the measurements below were made using the following minimal pipeline.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-bayer,format=bggr,bpp=10,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! ispclbayer10to8 ! queue ! ispclawb ! fakesink -v
Summary
In this summary you can find measurements related with CPU usage, memory, execution time and maximum framerate supported by the auto-white balance algorithm.
| Resolution | CPU usage (%) | Memory Bandwidth (GHz) | Execution Time (ms) |
|---|---|---|---|
| 640x480@30fps | 10 | 0.994 | 3.11 |
| 1280x720@30fps | 12 | 1.032 | 4.54 |
| 1920x1080@30fps | 16 | 1.239 | 8.52 |
| Resolution | Maximum framerate (fps) * |
|---|---|
| 640x480 | 365 |
| 1280x720 | 135 |
| 1920x1080 | 62 |
| Note: take into consideration that these values were taken using the videotestsrc element, and may decrease with a camera source capture. |
CPU usage
These measurements were taken with the top app to monitor the behavior of the pipeline while running.
Memory usage
These measurements were taken with perf stat linux tool to measure the bandwidth needed by the pipeline.
Execution time
These measurements were taken using some timestamps before and after the processing to measure just the execution time for the kernel algorithm on each frame and hence avoiding possible false measurements from the GStreamer plugin.
Maximum Framerate
These measurements were taken with the gst-perf tool to monitor the behavior of the pipeline while running. When using videotestsrc we are able to process as many buffers as the element can handle, hence the maximum value reported.
The following pipeline was used to test the maximum framerate, please note the use of videotestsrc.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 videotestsrc ! "video/x-bayer,format=bggr,width=$WIDTH,height=$HEIGHT,framerate=30/1" ! queue ! ispclawb ! perf ! fakesink -v
Histogram Equalization Benchmarking
The following section describes some relevant performance measurements for the Histogram Equalization for a few standard resolutions. It begins with a summary of the results detailed on the following sections, as well as the pipelines used to capture such results.
Pipeline
All the measurements below were made using the following minimal pipeline.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-bayer,format=bggr,bpp=10,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! ispclbayer10to8 ! queue ! ispcleq ! fakesink -v
Summary
In this summary you can find measurements related with CPU usage, memory, execution time and maximum framerate supported by the Histogram Equalization algorithm.
| Resolution | CPU usage (%) | Memory Bandwidth (GHz) | Execution Time (ms) |
|---|---|---|---|
| 640x480@30fps | 10 | 0.990 | 4.28 |
| 1280x720@30fps | 12 | 1.047 | 6.07 |
| 1920x1080@30fps | 16 | 1.277 | 10.95 |
| Resolution | Maximum framerate (fps)* |
|---|---|
| 640x480 | 345 |
| 1280x720 | 135 |
| 1920x1080 | 62 |
| Note: take into consideration that these values were taken using the videotestsrc element, and may decrease with a camera source capture. |
CPU usage
These measurements were taken with the top app to monitor the behavior of the pipeline while running.
Memory usage
These measurements were taken with perf stat linux tool to measure the bandwidth needed by the pipeline.
Execution time
These measurements were taken using some timestamps before and after the processing to measure just the execution time for the kernel algorithm on each frame and hence avoiding possible false measurements from the GStreamer plugin.
Maximum Framerate
These measurements were taken with the gst-perf tool to monitor the behavior of the pipeline while running. When using videotestsrc we are able to process as many buffers as the element can handle, hence the maximum value reported.
The following pipeline was used to test the maximum framerate, please note the use of videotestsrc.
$ WIDTH=1920 $ HEIGHT=1080 $ gst-launch-1.0 videotestsrc ! "video/x-bayer,format=bggr,width=$WIDTH,height=$HEIGHT,framerate=30/1" ! queue ! ispcleq ! perf ! fakesink -v
