RidgeRun Metadata/RidgeRun Metadata Catalog/MPEG-TS
 | The RidgeRun Metadata documentation from RidgeRun is presently being developed. |
Introduction
In-band metadata refers to additional information —such as GPS coordinates, telemetry, or timecodes— that travels in the same channel as the video, maintaining synchronization through shared timestamps. According to the Motion Imagery Standards Profile (MISP), metadata must be encoded in KLV (Key-Length-Value) format, which ensures interoperability across systems. KLV acts as a generic binary container where sender and receiver agree on the semantics of each value, while timestamps guarantee alignment with the correct video frame.
The MPEG-2 Transport Stream (TS) protocol natively supports multiple elementary streams (video, audio, metadata), using Packet Identifiers (PIDs) and built-in timestamp mechanisms. This makes it a robust standard for in-band metadata, allowing KLV messages to travel alongside video without breaking compatibility for receivers that ignore them, while enabling precise synchronization for those that interpret them.
RidgeRun developed a GStreamer meta-plugin that enables in-band metadata support in MPEG-TS. This plugin provides two helper elements:
- metasrc: injects metadata into the pipeline.
- metasink: extracts metadata buffers from the pipeline, allowing applications to receive and process incoming metadata.
The actual metadata support is implemented by RidgeRun’s extensions to mpegtsmux and tsdemux, which add synchronous and asynchronous KLV metadata insertion and extraction through a dedicated metadata pad. These extensions align with MISB standards and enable integration of metadata alongside MPEG-TS video and audio streams.
Supported Protocols and Containers
The MPEG-2 TS container is the standard carrier for in-band metadata. RidgeRun’s plugin is format-agnostic: it can inject metadata into the muxer regardless of the encoding format of the video (e.g., H.264, H.265).
Key scenarios include:
- MPEG-2 TS + H.264/H.265 – transport or storage with synchronized metadata.
- RTP over UDP – the metadata is first inserted into the MPEG-TS stream using
mpegtsmux, which can then be packetized withrtpmp2tpayand transmitted over UDP (e.g.,mpegtsmux ! rtpmp2tpay ! udpsink).
Note: with in-band metadata, you are limited to the MPEG-TS container; metadata is not directly supported in other container formats.
Benefits and Limitations
| Category | Item | Description |
|---|---|---|
| Benefits | Standards-based | Compliant with MISB and KLV, ensuring interoperability. |
| Frame-accurate synchronization | Metadata is timestamped and aligned with video frames. | |
| Compatibility | Receivers that do not parse metadata continue normal playback. | |
| Flexible injection | Metadata can be added through the dedicated metadata pad (by pushing buffers directly) or via the metasrc element, which simplifies the process.
| |
| Extended support | RidgeRun patches bring full synchronous/asynchronous metadata handling to mpegtsmux and tsdemux.
| |
| Commercial support | Many third-party players already recognize and extract KLV metadata carried in MPEG-TS, reducing the need for custom elements on the receiving side. | |
| Limitations | Container restriction | In-band metadata is limited to the MPEG-TS container. |
| Frame association | While metadata is timestamped for synchronization, its linkage to a specific video frame is not as direct as with SEI or OBU; applications must rely on timestamps for alignment. |
Examples
- Injecting periodic metadata into an H.264 transport stream
gst-launch-1.0 -e metasrc metadata=$TIME_METADATA period=1 ! 'meta/x-klv' ! mpegtsmux name=mux ! filesink sync=false async=true location=$FILE \ videotestsrc is-live=true ! 'video/x-raw,format=I420,width=320,height=240,framerate=30/1' ! x264enc ! mux.
- Extracting metadata alongside H.264 video
gst-launch-1.0 -e filesrc location=$FILE ! tsdemux name=demux \ demux. ! queue ! h264parse ! 'video/x-h264,stream-format=byte-stream,alignment=au' ! avdec_h264 ! autovideosink \ demux. ! queue ! 'meta/x-klv' ! metasink
Documentation
