OpenGL Accelerated HTML Overlay/Examples/GStreamer Usage: Difference between revisions

Latest revision as of 21:27, 1 October 2024

General

HTMLOverlay includes out of the box, an easy to use GStreamer plugin, that allows users to add overlays to media in a quick an easy manner.

GStreamer Element

Pre-requisites

NVIDIA Jetson

If you are on an NVIDIA Jetson, you may need to set the following variables

export DISPLAY=:0
export WEBKIT_DISABLE_COMPOSITING_MODE=1

htmloverlay

This element takes an RGBA video stream, renders the website and overlays the resulting website on top of the incoming video.

Example pipeline

On Jetson platforms:

gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! xvimagesink

On x86 platforms:
With video sources:

gst-launch-1.0 videotestsrc pattern=ball ! "video/x-raw,height=720,width=1280,framerate=30/1" ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! videoconvert ! xvimagesink

With camera to file:

gst-launch-1.0 v4l2src num-buffers=100 ! videoconvert ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! videoconvert ! x264enc ! h264parse ! qtmux ! filesink location=test.mp4 -v

Use cases

In this section, you can find some use cases for the HTMLOverlay. The examples were tested on a Jetson Xavier NX with Jetpack 5.1.1.

Recording

The following pipeline records a video from a camera sensor applies the three different algorithms, and saves the video into an MP4 file.

gst-launch-1.0 -ve nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mp4 -e

Streaming

The following pipeline is an example of a streaming application with the overlay element. These pipelines use another RidgeRun product for high-performance streaming called GstRtspSink.

gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! nvv4l2h264enc insert-sps-pps=true idrinterval=30 ! "video/x-h264,mapping=/stream1" ! rtspsink service=5000 -vvv

Previous: Examples/Library_Usage

Index

Next: Performance/NVIDIA_Jetson

@@ Line 1: / Line 1: @@
 <noinclude>
-{{OpenGL Accelerated HTML Overlay/Head|previous=|next=|metakeywords=Contact Us}}
+{{OpenGL Accelerated HTML Overlay/Head|previous=Examples/Library_Usage|next=Performance/NVIDIA_Jetson|metakeywords=GStreamer Usage}}
 </noinclude>
-{{UnderConstruction}}
 == General ==
-In the following sections, you can see examples of how to use them. You can see more information about the algorithms these elements apply in [[CUDA_ISP_for_NVIDIA_Jetson/CUDA_ISP_for_NVIDIA_Jetson_Basics|CUDA ISP Basics]]. Consider that a patch must be applied to <code>v4l2src</code> to enable captures in bayer10. You can see how to apply this patch in this link: [[CUDA_ISP_for_NVIDIA_Jetson/Getting_Started/Building_custom_v4l2src_element|Apply patch to v4l2src]].
+HTMLOverlay includes out of the box, an easy to use GStreamer plugin, that allows users to add overlays to media in a quick an easy manner.
 == GStreamer Element ==
-=== cudashift ===
+=== Pre-requisites ===
+==== NVIDIA Jetson ====
+If you are on an NVIDIA Jetson, you may need to set the following variables
+<pre>
+export DISPLAY=:0
+export WEBKIT_DISABLE_COMPOSITING_MODE=1
+</pre>
+=== htmloverlay ===
-This element applies the shifting algorithm. For more information about the element properties and capabilities, check here: [https://ridgerun.github.io/libcudaisp-docs/md__builds_ridgerun_rnd_libcudaisp_docs_cudashift.html cudashift element].
+This element takes an RGBA video stream, renders the website and overlays the resulting website on top of the incoming video.
-* '''Choosing the shift value'''
+* '''Example pipeline'''
-On Jetson platforms
+<u>'''On Jetson platforms:'''</u>
+<br>
 <source lang=bash>
-gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb algorithm=1 ! 'video/x-raw, format=RGB' ! fakesink
+gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! xvimagesink
 </source>
+[[File:Overlay jetson cam.mp4|800px|frameless|center|alt=GStreamer example using HTML Overlay]]
+<u>'''On x86 platforms:'''</u>
+<br>
+'''With video sources:'''
+<source lang=bash>
+gst-launch-1.0 videotestsrc pattern=ball ! "video/x-raw,height=720,width=1280,framerate=30/1" ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! videoconvert ! xvimagesink
+</source>
+[[File:Example simple overlay x86.mp4|800px|frameless|center|alt=GStreamer example using HTML Overlay]]
+<br>
+'''With camera to file:'''
+<source lang=bash>
+gst-launch-1.0 v4l2src num-buffers=100 ! videoconvert ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! videoconvert ! x264enc ! h264parse ! qtmux ! filesink location=test.mp4 -v
+</source>
+[[File:Webcam overlay.mp4|800px|frameless|center|alt=GStreamer example using HTML Overlay]]
 == Use cases ==
-In this section, you can find some use cases for the CUDA ISP GStreamer plugins. Both examples were tested on a Jetson Xavier NX.
+In this section, you can find some use cases for the HTMLOverlay. The examples were tested on a Jetson Xavier NX with Jetpack 5.1.1.
+*'''Recording'''
-*'''Recording'''
+The following pipeline records a video from a camera sensor applies the three different algorithms, and saves the video into an MP4 file.
-The following pipeline records a video from a camera sensor, applies the three different algorithms and saves the video into an MP4 file.
 <source lang=bash>
-GST_DEBUG=WARNING,*cudaawb*:LOG,*cudadebayer*:LOG gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudashift shift=5 ! cudadebayer ! cudaawb algorithm=2 confidence=0.80 ! 'video/x-raw,format=I420' ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mp4 -e
+gst-launch-1.0 -ve nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test.mp4 -e
 </source>
+*'''Streaming'''
-*'''Streaming'''
+The following pipeline is an example of a streaming application with the overlay element. These pipelines use another RidgeRun product for high-performance streaming called [[GstRtspSink]].
-The following pipeline is an example of a streaming application with the three elements. These pipelines use another RidgeRun product for high-performance streaming called [[GstRtspSink]] .
 <source lang=bash>
-GST_DEBUG=*:ERROR gst-launch-1.0 v4l2src do-timestamp=1 io-mode=userptr ! "video/x-bayer,bpp=10,width=3840,height=2160,framerate=30/1" ! cudashift shift=7 ! queue ! cudadebayer ! queue ! cudaawb algorithm=2 confidence=0.85 ! queue ! "video/x-raw,format=I420" ! nvvidconv ! queue ! nvv4l2h264enc insert-sps-pps=true idrinterval=30 ! "video/x-h264,mapping=/stream1" ! rtspsink service=5000 -vvv
+gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),height=1080,width=1920,framerate=30/1" ! nvvidconv ! queue ! htmloverlay url="http://0.0.0.0:8000/fancy_overlay.html" enable-js=true web-refresh-rate=0 ! queue ! nvvidconv ! nvv4l2h264enc insert-sps-pps=true idrinterval=30 ! "video/x-h264,mapping=/stream1" ! rtspsink service=5000 -vvv
 </source>
 <noinclude>
-{{OpenGL Accelerated HTML Overlay/Foot||}}
+{{OpenGL Accelerated HTML Overlay/Foot|Examples/Library_Usage|Performance/NVIDIA_Jetson}}
 </noinclude>