Gstreamer QT Overlay

 PAGE UNDER CONSTRUCTION

Overview

GstQtOverlay is a GStreamer plug-in that renders QT graphics on top of a video stream. This rendering occurs directly into the image buffer, rather than in physical graphics memory, which brings together the best of the two worlds:

• Modularity and extensibility of GStreamer
• Beauty and flexibility of QT

Now, GUIs will be available not only at the video display, but also on file recordings and network streaming, for example.

GstQtOverlay makes heavy use of OpenGL in order to achieve the overlay rendering with high performance. The pixels blit and memory transfers are done by the GPU and DMA accelerators, allowing real time operation at HD resolutions. As a consequence, the CPU remains free for other processing.

Graphics are modeled using QT's QML, which enables fast, powerful and dynamic GUI implementation. QML is loaded at runtime so there is no need to recompile the plug-in in order to change the GUI. This speeds up development time, eases quick prototyping and reduces time-to-market without hurting significantly performance. A single GstQtOverlay is fit for many different GUIs.

Architecture

GstQtOverlay's architecture is summarized in Figure 1. Each layer is describe further with more detail.

GstQtOverlay Architecture

• GStreamer

This layer provides the necesary classes to expose GstQtOverlay as a GStreamer element. As such, GstQtOverlay can be linked into any compatible pipeline and will participate of the standard pipeline lifecycle. This includes among other, caps and allocator negotiation.

• QT/QML

On top of Gstreamer, a QT powered class is instantiated in order to manage graphic resources in a decoupled way. Here, QT event loop is specially processed in order to have both Glib's and QT's event sources active. Finally, a QML engine is created in order to read and properly render the user's QML file.

• HAL

The Hardware Abstraction Layer (or simply HAL) is a thin layer that acts as an adapter between QT and OpenGL in order to leverage the platform specific GPU utilities and boost performance to its maximum. This layer is the only one that is HW dependent and will be conditionally built via configuration parameters. The HAL ensures that data transfers are HW accelerated so that there is no performance penalty.

• OpenGL

The OpenGL layer will render QT content into the image data using the GPU parallel power. This blit is performed into a copy of the image, so that the integrity of the original data is preserved.

Supported Hardware

Currently, the following hardware is properly supported:

• X86 - Linux Ubuntu 16.04
• X86 - Apple Mac OSX
• IMX6

More platforms are under development. If you wish to sponsor the port for a custom platform or accelerate development for another not in the list above, please contact us to support@ridgerun.com

FAQ

• Can I use external events to interact with my GUI?

Yes. See X

• Am I limited to a single QML?

No. As with any regular QML powered application, events can fire up the load of other QML files. This allows the user to build state-pattern like apps with ease.

• Can I use mouse and/or keyboard events?

Not yet. At least not out-of-the-box.

• Is QML enough for my GUI?

Yes (most likely). QML support is great, there's a lot of documentation and examples available.

• Can I create my own custom QML components?

Yes. See this guide.

• Can I use animations?

Not yet. We expect to support this in future releases

• Does my HW support GstQtOverlay?

Check X