V4L2 FPGA/Examples/Convolutioner: Difference between revisions
No edit summary |
(Add images, tables and more details) |
||
Line 14: | Line 14: | ||
That's not all! More optimizations are coming in our goal to achieve 30fps @4K. | That's not all! More optimizations are coming in our goal to achieve 30fps @4K. | ||
<gallery> | |||
'''Convolution I/O properties''' | |||
The input/output images have limitations in format and size, which are indicated below: | |||
{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;" | |||
|- | |||
! Property !! Input image !! Output image | |||
|- | |||
| Min width || 8 || 8 | |||
|- | |||
| Max width || 4096 || 4096 | |||
|- | |||
| Min height || 8 || 8 | |||
|- | |||
| Max height || 2160 || 2160 | |||
|- | |||
| Formats || 8-bit Gray (Mono) || 8-bit Gray (Mono) | |||
|} | |||
It is currently recommended for applications whose images are at 720p or 1080p resolution. | |||
'''Convolution in action''' | |||
Currently, the convolution example is fixed to a 3x3 Gaussian kernel, represented in 16-bit Fixed-Point (Q<sub>0,16</sub>): | |||
{| class="wikitable" caption="3x3 Gaussian Kernel" style="margin-left: auto; margin-right: auto; border: none;" | |||
|- | |||
| 0.0625 || 0.125 || 0.0625 | |||
|- | |||
| 0.125 || 0.25 || 0.125 | |||
|- | |||
| 0.0625 || 0.125 || 0.0625 | |||
|} | |||
The best example to show how a Gaussian blur works is to apply it to a solid line. According to the theory, the edges will degrade, losing their sharpness. This is illustrated in the following pictures: | |||
<gallery class="center"> | |||
Line.png|Input solid line | Line.png|Input solid line | ||
Convolution_Gauss.png|Line after convolution | Convolution_Gauss.png|Line after convolution | ||
</gallery> | </gallery> | ||
'''Current throughput''' | |||
The convolution accelerator has the following throughput in several resolutions: | |||
{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;" | |||
|+ align="top" style="caption-side: top" | Maximum framerate using 3x3 convolution for several standard resolutions on a PicoEVB | |||
|- | |||
! Resolution | |||
! Maximum framerate (fps) | |||
|- | |||
| 4k | |||
| 7.478 | |||
|- | |||
| 1080p | |||
| 29.485 | |||
|- | |||
| 720p | |||
| 64.972 | |||
|} | |||
For these measurements, we are using a 3x3 kernel. It is completely possible to have greater kernels without sacrificing speed, thanks to parallelism in terms of pixel calculation, which actually lasts one clock. Besides, having greater kernels will lead to more area consumption. | |||
<center>'''Do you need us to enhance this or building more complex accelerators?'''</center> | |||
<center> | |||
[[File:RR Contact Us.png|200px|link=http://www.ridgerun.com/contact]] | |||
</center> | |||
<noinclude> | <noinclude> | ||
{{V4L2_FPGA/Foot|Examples|Examples/Image Stitching}} | {{V4L2_FPGA/Foot|Examples|Examples/Image Stitching}} | ||
</noinclude> | </noinclude> |
Revision as of 17:38, 18 July 2019
V4L2 FPGA |
---|
Introduction |
Getting the Code |
Examples |
GStreamer Pipelines |
Supported Platforms |
Contact Us |
This accelerator is capable of receiving a video frame from the kernel space, apply a Gaussian blurring through a convolution in space (custom kernel values are planned for future releases) and return the frame to the kernel. This operation allows reducing the noise on an image and correct minor errors.
With multiple convolution accelerators, it is possible to perform more complex operations, such as demosaicing, Sobel, DoG (Differential of Gaussian), LoG (Laplacian of Gaussian), and other spatial filters. The code of the accelerator will be available after purchasing V4L2-FPGA and you can modify the code in order to implement other types of filters or increase the kernel size.
Hardware description optimizations allow this accelerator to avoid any bottleneck created by this module, thanks to data parallelism, fitting up to eight pixels in a single bus transference.
That's not all! More optimizations are coming in our goal to achieve 30fps @4K.
Convolution I/O properties
The input/output images have limitations in format and size, which are indicated below:
Property | Input image | Output image |
---|---|---|
Min width | 8 | 8 |
Max width | 4096 | 4096 |
Min height | 8 | 8 |
Max height | 2160 | 2160 |
Formats | 8-bit Gray (Mono) | 8-bit Gray (Mono) |
It is currently recommended for applications whose images are at 720p or 1080p resolution.
Convolution in action
Currently, the convolution example is fixed to a 3x3 Gaussian kernel, represented in 16-bit Fixed-Point (Q0,16):
0.0625 | 0.125 | 0.0625 |
0.125 | 0.25 | 0.125 |
0.0625 | 0.125 | 0.0625 |
The best example to show how a Gaussian blur works is to apply it to a solid line. According to the theory, the edges will degrade, losing their sharpness. This is illustrated in the following pictures:
-
Input solid line
-
Line after convolution
Current throughput
The convolution accelerator has the following throughput in several resolutions:
Resolution | Maximum framerate (fps) |
---|---|
4k | 7.478 |
1080p | 29.485 |
720p | 64.972 |
For these measurements, we are using a 3x3 kernel. It is completely possible to have greater kernels without sacrificing speed, thanks to parallelism in terms of pixel calculation, which actually lasts one clock. Besides, having greater kernels will lead to more area consumption.