GStreamer1.0 and V4L2 UserPtr
|
|
V4L2 User Pointer
V4L2 default behavior is to allocate its own buffer pool, capture an image in each buffer and provide that buffer to the user to consume. This, although convenient, is not always the desired use case, such as when the buffer needs to be allocated from a special memory pool. In such occasions, it is required that V4L2 fills user allocated buffers. This mode is known as User Pointer (USERPTR) streaming mode.
Consider the following scenario (pipeline simplified for reading simplicity):
gst-launch-1.0 v4l2src ! caps filter caps=video/x-raw,width=1920,height=1080,framerate=30/1 ! gloverlay ! glimagesink
- V4L2 source element will capture images from the camera
- GLOverlay will render an PNG on top of the camera frame (a logo for example)
- The overlayed frame will be rendered to a display by GLImageSink
It is important to study the memory requirements of the above pipeline. GLOverlay requires special GPU buffers. If provided otherwise, GLOverlay will allocate its own GPU buffers and data will be copied from the alien buffers into the GPU ones.
memcpy
.---- V4L2 buffer --------------------------------------> X ---> GPU buffer -.
gst-launch-1.0 v4l2src ! caps filter caps=video/x-raw,width=1920,height=1080,framerate=30/1 ! gloverlay ! glimagesink
There are many reasons to avoid this memory copy: performance, power consumption, latency, etc... This is where UserPtr comes into play.
.-------------------------------- GPU Buffer -----------------------------------------------------.
V .------------------------------------------------------------> GPU buffer -----------------. |
gst-launch-1.0 v4l2src io-mode=userptr ! caps filter caps=video/x-raw,width=1920,height=1080,framerate=30/1 ! gloverlay ! glimagesink
As it can be seen, now GLOverlay will allocate a GPU buffer, provide it to V4L2src who will fill it with an image and subsequently push it back to the GLOverlay. The memory copy has been avoided.
User Pointer not working in newer GStreamer-1.0
Gstreamer exposes V4L2 camera capture through its well known GstV4L2Src element. The user pointer mode can be enabled via the io-mode property.
io-mode : I/O mode
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
However, this seems to be broken in GStreamer-1.0. Specifically, the following message prints when calling the REQBUFS IOCTL in USERPTR mode:
0:00:02.005602011 2361 0xe1890 ERROR v4l2allocator gstv4l2allocator.c:771:gst_v4l2_allocator_start:<v4l2src0:pool:src:allocator> error requesting 2 buffers: Invalid argument
Strangely, testing a simpler, lower level application like YAVTA (Yet Another V4L2 Test Application) seems to work properly.
The Problem
The problem is that GstV4L2Src attempts to use a user space wrapper over V4L2 called libv4l2 if available (which is the case in most of the file systems). This library is a set of calls wrapping traditional V4L2 ioctls and exposing some additional features. This can be verified by noting that gstv4l2allocator.c calls v4l2_ioctl instead of the good old ioctl. However, by checking libv4l2 source code, the following shows:
case VIDIOC_REQBUFS:
{
struct v4l2_requestbuffers *req = arg;
/* IMPROVEME (maybe?) add support for userptr's? */
if (req->memory != V4L2_MEMORY_MMAP) {
errno = EINVAL;
result = -1;
break;
}
As it can be seen, USRPTR is not supported by this layer. YAVTA, on the other hand, does work because it calls the ioctl directly.
A Workaround
If you explore gst-plugins-good-1.9.1/sys/v4l2/v4l2_calls.h, around line 29 you’ll see something like:
#ifdef HAVE_LIBV4L2 # include <libv4l2.h> #else
You may get around this issue by avoiding the GStreamer elements from using libv4l2. To do so you’ll need to configure gst-plugins-good as:
./configure --without-libv4l2
This should get you past this issue.
Additional Considerations
Another typical problem when using User Pointer mode, is that V4L2 fails to queue the buffers provided by the user. The debug shows as the following:
0:00:04.677438294 6329 0x13ec00 ERROR v4l2allocator gstv4l2allocator.c:1251:gst_v4l2_allocator_qbuf:<v4l2src0:pool:src:allocator> failed queueing buffer 0: Invalid argument 0:00:04.677526886 6329 0x13ec00 ERROR v4l2bufferpool gstv4l2bufferpool.c:1127:gst_v4l2_buffer_pool_qbuf:<v4l2src0:pool:src> could not queue a buffer 0
This is typically related to the nature of the memory passed to V4L2. Many drivers seem to be unable of handling non-contiguous, virtual, Linux memory to capture images. V4L2 documentation says that drivers may handle virtual memory by copying the data into each individual page, however by experience, I can tell you most of them will simply fail. Long story short, you need to use aligned, physically contiguous buffers with V4L2 userptr. Indeed, YAVTA does a posix_memalign to allocate its userptr buffers, if you change it to a regular malloc it will fail.
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
