V4L2 FPGA - GStreamer Pipelines - Filter

V4L2 FPGA
Introduction
Overview Frame Grabber Sink Filtering Budget Estimator
Getting the Code
How to get the code Building and Installation Guide Running the demo
Examples
Convolutioner Pass Through Video Test Pattern Generator Xilinx OpenCV
GStreamer Pipelines
Filter Mode Frame Grabber
Supported Platforms
PicoEVB Zynq 7000 series Zynq MP-SoC series LiteFury
Contact Us

Overview

This wiki section describes how to use the V4L2-FPGA for communicating with a hardware accelerator, which receives an image, does some processing, and sends it back to the system. You can try using one of the following examples for performing these tests:

• Pass Through
• Convolutioner

To use them, you need to configure your FPGA before running any of the pipelines shown below.

Basically, you can implement the pipeline illustrated in Figure 1 using GStreamer, which is performed in the following section.

V4L2 common pipeline

Camera Pipeline

The following pipelines generate a video pattern, sends it to the hardware accelerator implemented on the FPGA and retrieves the result.

4k Resolution

WIDTH=", width=(int)3840"
HEIGHT=", height=(int)2160"

gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw $WIDTH $HEIGHT" ! perf ! fakesink sync=false -v &
gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw $WIDTH $HEIGHT" ! v4l2sink sync=false device=/dev/video2 -v

For seeing the result frame, you can change fakesink by xvimagesink or similar.

1080p Resolution

WIDTH=", width=(int)1920"
HEIGHT=", height=(int)1080"

gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw $WIDTH $HEIGHT" ! perf ! fakesink sync=false -v &
gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw $WIDTH $HEIGHT" ! v4l2sink sync=false device=/dev/video2 -v

Performance Summary

Table 1. Performance for 3x3 convolution on several standard resolutions running at 30fps on a PicoEVB

Resolution	CPU usage (%)	Memory (MB)	Memory Bandwidth Usage (%)
4k@7	2.13	60	1
1080p@30	3.75	25	1
720p@30	2.53	15	1

Table 2. Maximum framerate using 3x3 convolution for several standard resolutions on a PicoEVB

Resolution	Maximum framerate (fps)
4k	7.478
1080p	29.485
720p	64.972

Table 3. Performance for a passthrough on several standard resolutions running at 30fps on a PicoEVB

Resolution	CPU usage (%)	Memory (MB)	Memory Bandwidth Usage (%)
4k@30	2.625	60	4
1080p@30	2.93	25	1
720p@30	2.34	15	1

Table 4. Maximum framerate using a passthrough for several standard resolutions on a PicoEVB

Resolution	Maximum framerate (fps)
4k	38.90
1080p	144.5
720p	294.729

CPU Usage

These measurements were taken with the tegrastats app to see the behavior of the pipeline while running at 30fps.

Cpu usage for convolution for different resolutions at 30fps.

Cpu usage for passthrough for different resolutions at 30fps.

Memory Usage

These measurements were taken with tegrastats app to see the behavior of the pipeline while running at 30fps.

Memory usage for convolution for different resolutions at 30fps.

Memory usage for passthrough for different resolutions at 30fps.

Maximum Framerate

These measurements were taken with gstperf tool to see the behavior of the pipeline while running, with videotestsrc we are able to push buffers as much as possible to the element. The image below shows the maximum framerate that the debayer element supports. It depends on the execution time of the debayer kernel algorithm.

Maximum framerate for convolution for different resolutions.

Maximum framerate for passthrough for different resolutions.

Memory Usage

These measurements were taken with tegrastats file to see the behavior of the pipeline while running at 30fps.

Memory bandwidth consuption for convolution for different resolutions.

Memory bandwidth consuption for passthrough for different resolutions.