FPGA Image Signal Processor - FPGA ISP Accelerators - Interpolators
| FPGA Image Signal Processor |
|---|
 |
| Introduction |
| FPGA ISP Accelerators/Modules |
| Getting the Code |
| Examples |
| GStreamer Pipelines |
| Tested Platforms |
| Contact Us |
Introduction
The FPGA-ISP Interpolators are a series of modules to interpolate mono/multi-channel images with variable pixel-size. All the interpolations are done with fixed-point number representation, which leads to optimizing the usage of area and accelerator speed. Depending on the interpolator, the fixed-point representation is optimized in order to get the minimum area footprint.
To perform the interpolations, it is possible to choose between the following interpolators:
- Nearest Neighbor
- Bilinear (still under development)
- Truncation: truncates the pixel value to the integer value
Supported caps
The FPGA-ISP Interpolators are capable of managing the following image properties:
Nearest neighbor
Maximum input: 4096x2160 Maximum output: 4096x2160 Pixels-per-clock: 2 (monochannel and multichannel) Number of channels: monochannel, multichannel Formats: Any (the pixel size is templated) unless planar Pixel-arrangement support: interleaved
Bilinear
Maximum input: 2047x2047 Maximum output: 2047x2047 Pixels-per-clock: 2 (monochannel) Number of channels: monochannel Formats: Monochannel (the pixel width is templated) Pixel-arrangement support: interleaved
Truncation
Maximum input: 4096x2160 Maximum output: 4096x2160 Pixels-per-clock: 2 (monochannel and multichannel) Number of channels: monochannel, multichannel Formats: Any (the pixel size is templated) unless planar Pixel-arrangement support: interleaved
Algorithm overview
The interpolators are modules. You can configure them according to your application. The module I/O is presented below:
Input
- Image pointer: the reference to the RAM (BRAM if FPGA does not have one)
- Input/Output image dimensions: width/height in pixels
- Points stream: Qs15.1 (Nearest Neighbor), Qs12.4 (Bilinear) in X,Y coordinates
- Pixel-width: templated (up to 32 bits)
- Interpolator: templated (BILINEAR, NEAREST_NEIGHBOR, TRUNCATION)
Output
- Out stream: Depending on the Pixel-width
Public members
- Coordinate type: to represent the points
- PPCO: pixels-per-clock at the output
- Maximum width
- Maximum height
Algorithms
The algorithms are based on the generic interpolators:
- Nearest Neighbor
- Bilinear
- Truncation: truncates to the lower near integer.
Some of the equations have been simplified and tailored to the application.
Benchmarks
The benchmarks are based on the GTU accelerator:
| Resolution | Maximum framerate (GRAY8-NN) | Maximum framerate (GRAY8-BI) | Maximum framerate (ARGB) |
|---|---|---|---|
| 4k | 30.03 | N/A | 15.23 |
| 1080p | 111.23 | 7.53 | 56.3 |
| 720p | 231.45 | 16.70 | 121.46 |
These framerates are taken based on the following setup:
System: NVidia Jetson Xavier FPGA: PicoEVB (Artix 7 XC7A50T CSG325 -2l) OS: Ubuntu 18.04 PCI-e: v2.0 - 1 lane
Also, these configurations:
-- GRAY8 -- Interpolator: Nearest Neighbor (NN) and Bilinear (BI) Input: 256x144 -- ARGB -- Interpolator: Nearest Neighbor Input: 72x72
You can reproduce these results by using the following pipelines:
Generator (Accelerator input)
gst-launch-1.0 videotestsrc ! video/x-raw,format=width=256,height=144 ! v4l2sink device=/dev/video2 -v
Sink (Accelerator output)
gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw, width=640, height=480,format=GRAY8" ! perf ! fakesink sync=false
Known issues
1. GStreamer autonegotiation: The caps, such as width, height, and format, must be specified in the pipeline.
2. Numerical precision: Since the resolution is lower than using floating-point numbers, different results might be obtained.
References