V4L2 FPGA - Examples - Convolutioner

This accelerator is capable of receiving a video frame from the kernel space, apply a Gaussian blurring through a convolution in space (custom kernel values are planned for future releases), and return the frame to the kernel. This operation allows reducing the noise on an image and correct minor errors.

With multiple convolution accelerators, it is possible to perform more complex operations, such as demosaicing, Sobel, DoG (Differential of Gaussian), LoG (Laplacian of Gaussian), and other spatial filters. The code of the accelerator will be available after purchasing V4L2-FPGA and you can modify the code in order to implement other types of filters or increase the kernel size.

Hardware description optimizations allow this accelerator to avoid any bottleneck created by this module, thanks to data parallelism, fitting up to eight pixels in a single bus transference.

That's not all! More optimizations are coming in our goal to achieve 30fps @4K.

Convolution I/O properties

The input/output images have limitations in format and size, which are indicated below:

Property	Input image	Output image
Min width	8	8
Max width	4096	4096
Min height	8	8
Max height	2160	2160
Formats	8-bit Gray (Mono)	8-bit Gray (Mono)

It is currently recommended for applications whose images are at 720p or 1080p resolution.

Convolution in action

Currently, the convolution example is fixed to a 3x3 Gaussian kernel, represented in 16-bit Fixed-Point (Q_0,16):

0.0625	0.125	0.0625
0.125	0.25	0.125
0.0625	0.125	0.0625

The best example to show how a Gaussian blur works are to apply it to a solid line. According to the theory, the edges will degrade, losing their sharpness. This is illustrated in the following pictures:

Input solid line
Line after convolution

The result has been zoom in to highlight the results. It is possible to appreciate the blurring in the second image, which is the convolution output.

Current throughput

The convolution accelerator has the following throughput in several resolutions:

Maximum framerate using 3x3 convolution for several standard resolutions on a PicoEVB
Resolution	Maximum framerate (fps)
4k	7.478
1080p	29.485
720p	64.972

For these measurements, we are using a 3x3 kernel. It is completely possible to have greater kernels without sacrificing speed, thanks to parallelism in terms of pixel calculation, which actually lasts one clock. Besides, having greater kernels will lead to more area consumption.

Do you need us to enhance this or building more complex accelerators?

Previous: Examples

Index

Next: Examples/Pass Through