OpenGL Accelerated HTML Overlay: Examples - GStreamer Usage

Follow Us On

Previous: Examples/Library_Usage

Index

Next: Performance/NVIDIA_Jetson

General

HTMLOverlay includes out of the box, an easy to use GStreamer plugin, that allows users to add overlays to media in a quick an easy manner.

GStreamer Element

Pre-requisites

NVIDIA Jetson

If you are on an NVIDIA Jetson, you may need to set the following variables

export DISPLAY=:0
export WEBKIT_DISABLE_COMPOSITING_MODE=1

htmloverlay

This element takes an RGBA video stream, renders the website and overlays the resulting website on top of the incoming video.

Example pipeline

On Jetson platforms:

gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! xvimagesink

On x86 platforms:
With video sources:

gst-launch-1.0 videotestsrc pattern=ball ! "video/x-raw,height=720,width=1280,framerate=30/1" ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! videoconvert ! xvimagesink

With camera to file:

gst-launch-1.0 v4l2src num-buffers=100 ! videoconvert ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! videoconvert ! x264enc ! h264parse ! qtmux ! filesink location=test.mp4 -v

htmloverlaysrc

The htmloverlaysrc element does not receive an input buffer; instead, it pulls the web page and renders it into an empty buffer. Its purpose is to be mixed later with another video source using glvideomixer.

On Xilinx Ultrascale platforms:

Render the web page to the screen:

gst-launch-1.0 htmloverlaysrc url="http://0.0.0.0:8000/ridgerun_overlay.html" enable-js=true web-refresh-rate=5 num-buffers=100  width=1280 height=720 ! queue ! videoconvert ! xvimagesink

Generate the HTML overlay output: This pipeline takes the buffer generated by htmloverlaysrc and uses glvideomixer to blend it with an external video source, producing a video stream that overlays the HTML content.

gst-launch-1.0 -v -e \
  glvideomixer name=mix background=black \
    sink_0::zorder=0 \
    sink_1::zorder=1 sink_1::alpha=1.0 sink_1::xpos=0 sink_1::ypos=0 ! \
  video/x-raw\(memory:GLMemory\),format=RGBA,width=1280,height=720,framerate=30/1 ! \
  gldownload ! queue ! videoconvert ! \
  omxh264enc target-bitrate=4000000 control-rate=variable ! h264parse ! mp4mux faststart=true ! filesink location=out.mp4 \
  \
  videotestsrc pattern=smpte is-live=true do-timestamp=true num-buffers=300 ! \
    video/x-raw,format=RGBA,width=1280,height=720,framerate=30/1 ! \
    glupload ! queue ! mix.sink_0 \
  \
 htmloverlaysrc url="http://0.0.0.0:8000/ridgerun_overlay.html" enable-js=true web-refresh-rate=5 \
 ! queue ! videoconvert ! video/x-raw,format=RGBA ! glupload ! queue ! mix.sink_1

Use cases

In this section, you can find some use cases for the HTMLOverlay. The examples were tested on a Jetson Xavier NX with Jetpack 5.1.1.

Recording

The following pipeline records a video from a camera sensor applies the three different algorithms, and saves the video into an MP4 file.

gst-launch-1.0 -ve nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mp4 -e

Streaming

The following pipeline is an example of a streaming application with the overlay element. These pipelines use another RidgeRun product for high-performance streaming called GstRtspSink.

gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! nvv4l2h264enc insert-sps-pps=true idrinterval=30 ! "video/x-h264,mapping=/stream1" ! rtspsink service=5000 -vvv