libguvc: USB Video Class Gadget Library for Linux
|
|
|
libGuvc: USB Video Class Gadget Library for Embedded Linux
UVC Gadget Development with libGuvc: A USB Video Class Library for Embedded Linux and libGuvc
Do you want to make your embedded device look like a webcam and stream video over USB to a computer? If yes, then you need to implement the UVC specification.
UVC or USB Video Class is a USB specification created by the USB Implementers Forum (USB-IF), and it is intended to standardize the video streaming functionality on the USB. That is, if you want to create a USB embedded device capable of sending video over the USB port and compatible with most operating systems and applications, then you need to implement the UVC specification.
Linux-based operating systems, as well as others, have support for UVC-based devices, such as webcams, acting as a host for the device. In the case of the device side, also known as the gadget, the driver support is usually proprietary, depending on the manufacturer. The Linux kernel has added an implementation of a UVC gadget driver to its mainline to help developers create Linux-based devices with UVC support.
Besides those efforts, the current UVC driver does not implement all the UVC specification, and it is quite hard to configure and use, depending heavily on a userspace application in order to complete the enumeration process and start the video streaming. For more information on how to configure and use the UVC driver, please refer How to use the UVC gadget driver in Linux.
The USB Video Class Gadget Library, or 
for short, is a platform-agnostic library that simplifies the development of UVC-based gadget devices by encapsulating most of the UVC communication, leaving just the basic setup to the user. It runs on top of the standard UVC driver in the Linux kernel and exposes a friendly interface for the userspace application, taking care of the communication between the user application and the Linux driver stack. Figure 1 depicts the software stack on a common use case scenario.
libGuvc Features by Release Version
The following table lists the main features provided by the libguvc and the corresponding library version:
| Feature | Release version |
|---|---|
| Isochronous endpoint support | 1.0.x |
| User Pointer support | 1.0.x |
| Extension Units support | 1.0.x |
| YUY2 video format support | 1.0.x |
| USB requests support | 1.0.x |
| Pull mode support | 1.0.x |
| MMAP support | 1.1.x |
| MJPEG support | 1.1.x |
| gst-uvc-sink element | 1.2.x |
| H264 support* | 1.3.x |
| Bulk endpoint support | 1.4.x |
- The libGuvc library only implements the UVC 1.1 specification since the Linux kernel driver only implements UVC 1.1. Future support for UVC 1.5 can be added after the Linux kernel driver adds support.
A Platform-Agnostic Library
The libGuvc library can be integrated into any embedded device, and it is independent of the SoC architecture and kernel version. It only requires the UVC gadget driver to be enabled in the platform.
For the time being, we have tested the library on the following platforms:
| Tested Environments | |||||
|---|---|---|---|---|---|
| SoC | Kernel Version | ||||
| NXP i.MX6 | 3.14 | ||||
| NXP I.MX8MM | 4.14 / 5.4 | ||||
| NXP I.MX8MQ | 5.15 | ||||
| NVIDIA Jetson | 4.9 / 5.x | ||||
| Qualcomm Snapdragon 410 | 4.14 | ||||
| Qualcomm RB5 | 5.15 | ||||
If you are interested in testing it on a different platform, please contact our highly-skilled embedded engineering services team today.
Note: libGuvc uses the kernel's UVC gadget driver as is; to demonstrate how to add support for features such as H264 or XU, kernel patches are needed and Ridgerun has experience creating them, they are not intended to fix any issues (if any) on the UVC driver or the platform's USB stack.
Independent of the Host OS
The libGuvc uses the Linux UVC gadget driver and implements the UVC standard so that the gadget device using the library can be used on most operating systems that support the UVC standard, such as Linux, Windows*, and Mac OS X. Likewise, it can be used by most standard capture applications such as Skype, VLC, guvcview, among others.
* For Windows, it is required to create a custom Windows INF file for the device to be successfully registered on the system
Independent of User Application Implementation
The libGuvc library can be integrated with any user application and only requires being fed with video frames that will be sent to the USB endpoint. This design allows it to easily integrate the library with any video frame wrappers, such as V4L2 buffers or GStreamer buffers, taking care only of the real video content and allowing the user to specify the memory management for the buffer types used.
GStreamer Integration
On release 1.2.x the library includes a sample GStreamer element that can be used to send UVC driver data directly to a host computer on a simple GStreamer pipeline, speeding up the development time for a UVC-based application. You can use this element as a starting point for your own GStreamer elements and add additional features.
User Pointer and MMAP Support
libGuvc can be used on User Pointer or MMAP mode in order to increase the compatibility with the userspace applications.
Extension Units Support
As part of the key features of the libGuvc, there is the support for UVC Extension Units (XU) allowing the user to implement custom controls in order to meet specific design requirements with a minimum amount of effort.
USB Requests Support
libGuvc allows the installation of callback functions on the user application in order to receive any USB request on the controls exposed; this provides a direct communication between the host computer and the user application.
Multiple Video Formats Supported
In version 1.0. x, the libGuvc provides support for YUY2 video format only. Releases 1.1.x and 1.3.x provide support for MJPEG and H264 video formats as well.
USB 2.0 / USB 3.0 Support
Since libGuvc works on top of the Linux UVC driver, it can work on USB 2.0 and USB 3.0 interfaces as long as its support is provided for the USB driver stack in the target platform.
How Does libGuvc Work?
libGuvc is distributed as an automake project that generates a library that can be externally used for another application to add UVC streaming capabilities. Since libGuvc uses the UVC kernel driver, it uses the V4L2 events interface to communicate. Each event is mapped to specific callbacks and methods within the library and the user application to ensure intercommunication.
'libGuvc API is quite simple - the following pseudo code depicts the basic sequence used on a typical user application. libGuvc API allows the user to interact with the UVC driver using just a set of methods and callbacks, abstracting all the complexity of driver interaction and configuration.
#include <libguvc.h>
uvc_device * device;
void release_buffer (void **entity, void *args)
{
/* Release the buffer data that has been sent through the UVC */
}
void fill_uvc_buffer (unsigned long *data_addr, int *data_size, int *bytes_used, void **entity, void *args)
{
/* send the video data and custom information to the library */
}
int main (int argc , char **argv)
{
...
uvc_init (device, "/dev/video0");
...
/* Set the callbacks to pass and release the video content */
uvc_set_pull_data_handler (device, fill_uvc_buffer, NULL);
uvc_set_release_data_handler (device, release_buffer, NULL);
...
uvc_start (device);
...
uvc_close (device);
return 0;
}
Additionally, the API provides support for a wide range of configurable settings to use specific features such as Extension Units or UVC control requests.
Can I Use libGuvc with Another Gadget Driver?
Since libGuvc uses the standard Linux driver for UVC, it works like any other USB gadget driver in the system. This means that you can set it up as a composite driver to get the UVC function working alongside other gadget functions such as UAC (audio over USB), mass storage, RNDIS, etc. For more information about USB gadgets and composite drivers, see our guide on USB gadget composite drivers.
Given that, using libGuvc with another gadget driver only requires creating a composite driver containing your desired functions and using each function independently in your userspace application. For example, please consider Figure 2 for a composite driver containing UVC and UAC functions. At the lower layer, a composite driver contains both functions, UVC and UAC. This exposes a /dev/videoX device for the UVC driver and a sound card for the UAC driver. Then, the user can stream video directly to the video node using libGuvc and at the same time, use the audio card to stream or record audio.
libGuvc Dependencies
LibGUVC needs the following dependencies to be fulfilled:
sudo apt install -y automake sudo apt install -y build-essential
If you want to use LibGUVC with GStreamer, fulfill the following dependencies:
sudo apt install \ gstreamer1.0-x \ libgstreamer1.0-dev \ libgstreamer-plugins-base1.0-dev \ gstreamer1.0-plugins-bad \ libgstreamer-plugins-bad1.0-dev
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
