PC based example GStreamer pipelines with GstQtOverlay

1. Display a text overlay

For this example, create a file named: "main.qml" with the following content:

import QtQuick 2.0

Item {
  id: root
  transform: Rotation { origin.x: root.width/2; origin.y: root.height/2; axis { x: 1; y: 0; z: 0 } angle: 180 }
  
  Text {
     text: "Way cool QT imaging with GStreamer!"
     font.pointSize: 30
     color: "Black"
     objectName: "labelMain"
  }
}

gst-launch-1.0 videotestsrc ! qtoverlay qml=main.qml ! videoconvert ! ximagesink

This will result on a window like on Figure 1.

Figure 1. Simple example with label

2. Animated gif overlay

For this example, create a file named: "animation.qml" with the following content:

import QtQuick 2.0

Item {
  id: root
  transform: Rotation { origin.x: root.width/2; origin.y: root.height/2; axis { x: 1; y: 0; z: 0 } angle: 180 }
    Rectangle {
        width: animation.width; height: animation.height + 8
        AnimatedImage { id: animation; source: "animation.gif" }
        Rectangle {
            property int frames: animation.frameCount
            width: 4; height: 8
            x: (animation.width - width) * animation.currentFrame / frames
            y: animation.height
            color: "red"
        }
    }
 }

Then generate or download an animation ".gif" file and rename it as "animation.gif". This file will be used as the animation to be displayed from the QML file.

gst-launch-1.0 videotestsrc ! "video/x-raw, width=1280, height=720" ! qtoverlay qml=animation.qml ! videoconvert ! ximagesink

This is the result:

Figure 2. Simple example with animated gif

3. Using GstQtOverlay within a GStreamer application

This use case is applicable when using a pipeline within a GStreamer application, where the application should interact with the QML nodes to change them dynamically. For achieving it, please, take into account the qml-attribute.

1. Supposing you have the following QML file:

import QtQuick 2.0

Item {
  id: root
  transform: Rotation { origin.x: root.width/2; origin.y: root.height/2; axis { x: 1; y: 0; z: 0 } angle: 180 }
  
  Text {
     text: "Original!"
     font.pointSize: 30
     color: "Black"
     objectName: "labelMain"
  }
}

Please, notice that the objectName is set to "labelMain". It works as an ID to identify the nodes within the QML file. These IDs should be unique within the file to avoid ambiguity.

2. You can create a QtOverlay element with the following instructions:

GstElement overlay;
const gchar *factory = "qtoverlay";
overlay = gst_element_factory_make (factory, "overlay");

In the factory make, you need to pass the element name (findable by gst-inspect) and the name of the instance within the pipeline, respectively. After that, you can add it to a GstBin or a pipeline. This example can be easily applied to the NVIDIA Jetson platform. The differences will rely on the pipeline description within the application.

3. For dynamically changing the QML node "labelName", we can use the qml-attribute property. Reminding the syntax:

<item name>.<attribute name>:<value>

For changing the text of the node, it would be:

"labelMain.text:GStreamer with QT imaging is way cool!"

Programatically in the GStreamer application:

g_object_set (overlay, "qml-attribute", "labelMain.text:GStreamer with QT imaging is way cool!", NULL);

For example take this simple code segment:

import sys
import gi
import time
import random

from datetime import date
from datetime import datetime

gi.require_version('Gst', '1.0')

from gi.repository import GLib, Gst

Gst.init(sys.argv)


class Pipeline:
    def __init__(self) -> None:
        self.overlay = None

    def pipeline_init(self):
        pipeline = "videotestsrc ! video/x-raw,height=500,width=600 ! qtoverlay qml=main.qml name=my_overlay ! videoconvert ! ximagesink"
        pipeline = Gst.parse_launch(pipeline)
        self.overlay = pipeline.get_by_name('my_overlay')
        pipeline.set_state(Gst.State.PLAYING)

pipe = Pipeline()
pipe.pipeline_init()

stamp = date.today().strftime("%d/%m/%y")
pipe.overlay.set_property("qml-attribute", "label_date.text:{}".format(stamp))

while True:
    input("Update?")
    now = datetime.now().strftime("%H:%M:%S")
    pipe.overlay.set_property("qml-attribute", "label_time.text:{}".format(now))
    x_pos = random.random()
    y_pos = random.random()
    value = "{:.2f},{:.2f}".format(x_pos, y_pos)
    pipe.overlay.set_property("qml-attribute", "label_pos.text:{}".format(value))

Using the previous main.qml file, we get something like this:

Figure 3. Simple example with property change on runtime

4. From file with extra animations

For this example the same previous overall logic is used but with two major changes.

Pipeline:

filesrc location=bbb_sunflower_1080p_60fps_normal.mp4 ! qtdemux name=demux demux.video_0 ! h264parse ! avdec_h264 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,height=500,width=600 ! qtoverlay qml=main.qml name=my_overlay ! videoconvert ! ximagesink

Control code:

toggle = True

counter = 9

while True:
    time.sleep(1)
    now = datetime.now().strftime("%H:%M:%S")
    pipe.overlay.set_property("qml-attribute", "label_time.text:{}".format(now))
    x_pos = random.random()
    y_pos = random.random()
    value = "{:.2f},{:.2f}".format(x_pos, y_pos)
    pipe.overlay.set_property("qml-attribute", "label_pos.text:{}".format(value))
    pipe.overlay.set_property("qml-attribute", "dot.color:{}".format("red" if toggle else "transparent"))
    toggle = not toggle
    if counter == 6:
        pipe.overlay.set_property("qml-attribute", "b3.color:{}".format("transparent"))
    elif counter == 2:
        pipe.overlay.set_property("qml-attribute", "b2.color:{}".format("transparent"))
    elif counter == 0:
        pipe.overlay.set_property("qml-attribute", "b1.color:{}".format("transparent"))
    elif counter < 0:
        pipe.overlay.set_property("qml-attribute", "b0.color:{}".format("red" if toggle else "white"))
    counter = counter - 1

Figure 4. Example with property change on runtime over more elements

5. Stream video over network from RSTP source

Let's assume:

SOURCE_ADRESS=rtsp://10.251.101.176:5004/test: origin rtsp video

DEST_ADRESS=10.251.101.57: destination computer/board

PORT=5004: source/dest port

On destination computer/board:

gst-launch-1.0 udpsrc buffer-size=57600 caps="application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)JPEG\,\ a-framerate\=\(string\)9.000000\,\ payload\=\(int\)26\,\ ssrc\=\(uint\)1563094421\,\ timestamp-offset\=\(uint\)3492706426\,\ seqnum-offset\=\(uint\)11616" ! rtpjpegdepay ! jpegparse ! avdec_mjpeg ! videoconvert ! xvimagesink -v sync=false async=false

Previous: Examples

Index

Next: Examples/NXP

❯