V4L2 PCIe - List of Xilinx OpenCV functions compatible with V4L2_FPGA
| V4L2 FPGA |
|---|
 |
| Introduction |
| Getting the Code |
| Examples |
| GStreamer Pipelines |
| Supported Platforms |
| Contact Us |
Introduction
V4L2 FPGA project has an interface to use Xilinx OpenCV (xfOpenCV) to develop custom image processing accelerators. Most of them are compatible and meet the following requirements:
- Sequential access to the data (sequential consumption)
- Single streams (or multiple streams consumed at the same time)
If those conditions are not met, the compatibility is limited and may lead to deadlocks.
In this wiki, we list the functions available in Xilinx OpenCV 2018.3. New functions added later may be incompatible given that Vivado 2019.1 doesn't offer backwards compatibility.
List
List of compatible functions from Xilinx OpenCV 2018.3:
| Function | Compatible | Observations |
|---|---|---|
| rgba2yuv4 | No | It uses three different streams for Y, U, V channels. Since they are not consumed at the same time, it leads to deadlocks. |
| HOGDescriptor | No | It streams out without taking care of the order. The host should now how the accelerator streams out the results |
| warpTransform | To check | It uses BRAM/URAM storage to cache the access, because of the behaviour of the accelerator. It may lead to limited support. |
| equalizeHist | No | It needs two redundant streams, leading to a deadlock. Since they are not consumed at the same time, it leads to deadlocks. |
| Canny related functions | No | Xilinx discourages its usage in Vivado HLS |
| EdgeTracing (Canny) | No | Xilinx discourages its usage in Vivado HLS |
| densePyrOpticalFlow | No | Xilinx discourages its usage in Vivado HLS |
| DenseNonPyrLKOpticalFlow | No | Xilinx discourages its usage in Vivado HLS |
| Accumulate | Yes | It needs more than one stream, but it reads the streams at the same time. |
| accumulateSquare | Yes | |
| accumulateWeighted | Yes | |
| bilateralFilter | Yes | |
| boxFilter | Yes | The internal kernel is compatible |
| merge (Channel combine) | Yes | The internal kernel is compatible |
| extractChannel | To check | The stream is consumed completely. |
| colorthresholding | Yes | It uses HSV channels. |
| convertTo | Yes | |
| cornersImgToList | Yes | Warning: it consumes all the stream |
| cornerUpdate | Yes | The flow vectors should be small, since they should be mapped as memory and the support is limited. |
| filter2D | Yes | |
| Color Space Conversion | Yes | Consider that the planar cases use different streams. This may lead to incompatibilities. |
| demosaicing | Yes | |
| dilate | Yes | |
| duplicateMat | Yes | Warning: It should be in a dataflow and consumed at the same time. |
| erosion | Yes | |
| GaussianFilter | Yes | |
| calcHist | Yes | Warning: it consumes all the stream |
| HoughLines | Yes | Warning: it consumes all the stream |
| inRange | Yes | |
| integral | Yes | |
| KalmanFilter | Yes | It is independent from the image |
| xFMagnitude | Yes | Warning: It need redundant streams, but consumed in the same clock cycle |
| MeanShift | No | No sequential accesses |
| medianBlur | Yes | Its structure is similar to the Gaussian |
| OtsuThreshold | Yes | Warning: it consumes all the stream |
| paintmask | Yes | |
| reduce | Yes | |
| remap | Yes | It makes possible to use URAM |
| RGB2HSV | Yes | This only works for single pixel streams |
| scale | Yes | |
| Sobel | Yes | Warning: It splits the application into two matrices (x, y) |
| sum | Yes | Warning: it consumes all the stream |
| Threshold | Yes | |
| warpTransform | Yes | |
| warpAffine | To Verify | |
| warpPerspective | No | Non-sequential access |
| SVM | No | It reads from an offset |
| InitUndistortRectifyMapInverse | Yes | The members should be mapped into Memory, which may lead to limitations. |
| xFFindStereoCorrespondenceLBMNO | Verify | It requires memory, which may lead to limitations |
Known issues found during this implementation
We found some issues related to the integration of xfOpenCV kernels to the V4L2 FPGA project. We list them below:
1. Numerical representation: given the fact the xfOpenCV uses fixed-point numbers instead of floating-point, the final results differ.
2. New release incompatibilities: The V4L2 FPGA is based on Vivado 2018.2. The latest xfOpenCV release is based on Vivado 2019.1, which is not compatible backwards. In order to use the kernels from the library, please, use the release for 2018.x.
3. Timing issue in Gaussian Blur: The GaussBlur implementation in xfOpenCV has already issues related to the timing inside of the kernel. We made some fixes in order to get them work.
4. Kernel incompatibilities: some cores which doesn't have sequential access to the elements could lead to issues of incompatibilities with the AXI Stream, resulting in accelerator hangings.
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
