GStreamer In-Band Metadata for MPEG Transport Stream - Examples - GstLaunch

GStreamer In-Band Metadata for MPEG Transport Stream
MPEG TS Metadata Basics
Getting Started
Evaluating the project Getting the code Building natively Building the project
User Guide
Examples
Demo GstLaunch Python Gstd C/C++ RTSP stream with metadata
Performance
FAQ
Contact Us

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help .

Check the Installation

This page contains a series of command line examples that use gst-launch-1.0 to run GStreamer pipelines along with the meta plugin, please check that the meta plugin is correctly installed on your system with gst-inspect.

Command:

gst-inspect-1.0 meta

Expected output:

Plugin Details:
  Name                     meta
  Description              Elements used to send and receive metadata
  Filename                 /home/$USER/gst_$VERSION/out/lib/gstreamer-1.0/libgstmeta.so
  Version                  1.11.0
  License                  Proprietary
  Source module            gst-plugin-meta
  Binary package           RidgeRun elements
  Origin URL               http://www.ridgerun.com

  metasrc: Metadata Source
  metasink: Metadata Sink

  2 features:
  +-- 2 elements

Some variables will store data to be used as input in the metadata property on metasrc.

Variables:

export METADATA='hello_world'
export TIME_METADATA=The_current_time_is:%T
export FILE=test.ts
export TEXT=meta.txt

Simple Metadata Access

The next pipeline stores inject the metadata value in the metasrc property into a text file. The value will be inserted one time.

gst-launch-1.0 -e metasrc metadata=$METADATA period=1000 ! 'meta/x-klv' ! filesink sync=false async=true location=$TEXT

The metasrc element has a property to insert metadata periodically in the pipeline; this example sends time-format metadata periodically and stores it in a text file.

gst-launch-1.0 -e metasrc metadata=$TIME_METADATA period=1000 ! 'meta/x-klv' ! filesink sync=false async=true location=$TEXT

The next pipeline uses a metasink element for dumping the metadata content of the text file to the standard console output.

gst-launch-1.0 -e filesrc location=$TEXT ! 'meta/x-klv' ! metasink

H264

The following pipelines show that it's possible to inject (periodically or not) regular metadata into an H.264 video encoding and store both metadata and video in the same transport stream file.

gst-launch-1.0 -e metasrc metadata=$TIME_METADATA period=1000 ! 'meta/x-klv' ! mpegtsmux name=mux ! filesink sync=false async=true location=$FILE videotestsrc is-live=true ! \
'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x264enc ! mux.

Using metasink, you are able to extract the incoming metadata and show it in the standard output along with the H264 video frames.

gst-launch-1.0 -e filesrc location=$FILE ! tsdemux name=demux demux. ! queue ! h264parse ! 'video/x-h264, stream-format=byte-stream, alignment=au' ! \
avdec_h264 ! autovideosink demux. ! queue ! 'meta/x-klv' ! metasink

Client-server pipelines over the network can also be used along with metadata. The server pipeline has H264 video encoding and streaming over the network, along with periodic metadata.

H264-Server:

gst-launch-1.0 -e metasrc period=1000 metadata=$METADATA ! 'meta/x-klv' ! mpegtsmux name=mux ! udpsink host=127.0.0.1 port=3001 \
videotestsrc is-live=true ! 'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x264enc ! mux.

The client pipeline allows the H264 video decoding of the incoming stream to be displayed in the respective video, while the metasink element dumps the metadata content in the console standard output.

H264-Client:

gst-launch-1.0 -e udpsrc port=3001 ! 'video/mpegts, systemstream=(boolean)true, packetsize=(int)188' ! tsdemux name=demux demux. ! queue !  \
h264parse ! 'video/x-h264, stream-format=byte-stream, alignment=au' ! avdec_h264 ! autovideosink sync=false demux. ! queue ! 'meta/x-klv' ! metasink async=false

The following pipeline shows the insertion of synchronous metadata.

Server:

gst-launch-1.0 metasrc metadata=%T period=1000 ! meta/x-klv,parsed=true,stream_type=21 ! queue ! \
mpegtsmux name=mux ! filesink location=meta.ts videotestsrc is-live=true ! queue ! x264enc key-int-max=30 ! h264parse ! mux. -v

Client:

gst-launch-1.0 filesrc location=meta.ts ! queue ! tsdemux name=demux  ! queue ! h264parse ! avdec_h264 ! queue ! \
videoconvert ! autovideosink demux. ! queue ! meta/x-klv,parsed=true ! metasink -v

H265

The format H265 can also be used with in-band metadata, the pipelines follow a similar structure than H264.

gst-launch-1.0 -e metasrc metadata=$METADATA period=1000 ! 'meta/x-klv' ! mpegtsmux name=mux ! filesink sync=false async=true location=$FILE videotestsrc is-live=true ! \
'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x265enc ! mux.

The element metasink extracts the metadata and dumps it on the standard output.

gst-launch-1.0 -e filesrc location=$FILE ! tsdemux name=demux demux. ! queue ! h265parse ! 'video/x-h265, stream-format=byte-stream, alignment=au' ! \
avdec_h265 ! autovideosink demux. ! queue ! 'meta/x-klv' ! metasink

Client-server pipelines over the network, similar to H264 can also be used along with metadata. The server pipeline has H265 video encoding and streaming over the network along with periodic metadata.

H265-Server:

gst-launch-1.0 -e metasrc period=1000 metadata=$METADATA ! 'meta/x-klv' ! mpegtsmux name=mux ! udpsink host=127.0.0.1 port=3001 \
videotestsrc is-live=true ! 'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x265enc ! mux.

The client pipeline handles the incoming video streaming, displays the video buffers, and dumps the metadata to the standard output.

H265-Client:

gst-launch-1.0 -e udpsrc port=3001 ! 'video/mpegts, systemstream=(boolean)true, packetsize=(int)188' ! tsdemux name=demux demux. ! queue !  \
h265parse ! 'video/x-h265, stream-format=byte-stream, alignment=au' ! avdec_h265 ! autovideosink sync=false demux. ! queue ! 'meta/x-klv' ! metasink async=false

Custom pad

You can also request the meta pad in the mpegtsmux, in the following way:

gst-launch-1.0 udpsrc port=5610 ! tsdemux !  meta/x-klv,parsed=true ! mux.sink_0 mpegtsmux alignment=7 name=mux  ! udpsink host=xxx.xxx.xxx.xxx port=5000 \ 
videotestsrc is-live=true ! video/x-raw,width=100,height=100,framerate=30/1 ! x264enc ! queue ! mux.

This will let you have completely asynchronous video and metadata streaming even if one does not exist, without using a metasrc.

Faster metadata

To demonstrate a use case for faster metadata, the following pipeline will record a video at 30FPS (around 33ms) while writing the time (HH:MM:SS) at approximately the same frequency as the video, in parallel:

export TIME_METADATA=%T:%f
gst-launch-1.0 -e metasrc metadata=$TIME_METADATA period=33 ! 'meta/x-klv' ! queue ! mux.sink_%d mpegtsmux name=mux ! filesink 'sync=true' 'async=true' 'location=test.ts' videotestsrc 'is-live=true' ! 'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x264enc ! queue ! mux.

When building your own pipeline for faster metadata, it's important to include mux.sink_%d next to the mpegtsmux.

You can check that both the metadata and video are being recorded at a similar speed by using gst-perf:

gst-launch-1.0 -e metasrc metadata=$TIME_METADATA period=33 ! 'meta/x-klv' ! queue ! perf name=meta ! mux.sink_%d mpegtsmux name=mux ! filesink 'sync=true' 'async=true' 'location=test.ts' videotestsrc 'is-live=true' ! 'video/x-raw,format=(string)I420,width=320,height=240,framerate=(fraction)30/1' ! x264enc ! queue ! perf name=video ! mux.

To read the video back from the saved file, there are different alternatives, shown in order from the most recommendable to the least recommendable:

• Increase the queue limits of max-size-time and max-size-buffers until the pipeline starts running (lower metasrc periods may require higher max-size-* in the metasink queue):

gst-launch-1.0 filesrc location=test.ts ! tsdemux name=demux demux. ! queue ! h264parse ! 'video/x-h264, stream-format=byte-stream, alignment=au' ! avdec_h264 ! videoconvert ! xvimagesink demux. ! queue max-size-time=4000000000 max-size-buffers=200 ! 'meta/x-klv' ! metasink

• Disable the queue limits of max-size-time and max-size-buffers:

gst-launch-1.0 filesrc location=test.ts ! tsdemux name=demux demux. ! queue ! h264parse ! 'video/x-h264, stream-format=byte-stream, alignment=au' ! avdec_h264 ! videoconvert ! xvimagesink demux. ! queue max-size-time=0 max-size-buffers=0 ! 'meta/x-klv' ! metasink

• Use 'async=false' in the sink elements of the pipeline and 'sync=false' in metasink:

gst-launch-1.0 filesrc location=test.ts ! tsdemux name=demux demux. ! queue ! h264parse ! 'video/x-h264, stream-format=byte-stream, alignment=au' ! avdec_h264 ! videoconvert ! xvimagesink 'async=false' demux. ! queue ! 'meta/x-klv' ! metasink 'sync=false' 'async=false'

In case you want to measure the performance of these pipelines as well, use the perf name=some_name element after each queue