GStreamer Buffer Synchronization: Documentation - GStreamer

Introduction

To ease the integration of the RR Syncer synchronization engine, a GStreamer plug-in is included to perform frame-accurate buffer alignment across multiple video streams. The element relies on buffer timestamps and a shared synchronization context to guarantee temporal consistency in multi-camera pipelines.

Thanks to this design, the RR Syncer GStreamer element is able to compensate for stream latencies and deliver aligned buffers with low overhead. Synchronization is event-driven and supports callbacks for drop, flush, and sync events.

This wiki entry covers supported formats, features, and plugin properties.

Enabling the Plug-in

To enable the plug-in, compile the RR Syncer library with GStreamer support using:

To enable the plug-in, make sure to have compiled the GStreamer support in the library. By default, it should have been enabled (see Building the Project).

After the installation, please check the element by using:

gst-inspect-1.0 rrsyncer

The gst-inspect output will show information about pads, formats, properties, and expected behavior.

Supported Formats

The RR Syncer element supports raw video streams with the following format:

Input: video/x-raw (e.g., NV12, RGB, UYVY)
Output: Matches input format

Image dimensions and framerates should match across streams to ensure synchronization stability.

Features

The GStreamer plug-in included in the RR Syncer library provides robust synchronization of buffers from multiple video streams. The element allows fine-tuned configuration and is suitable for embedded and high-performance use cases. Here is a list of features:

Timestamp-Based Synchronization: Buffers are aligned using their `pts` (presentation timestamps). The synchronization algorithm ensures deterministic behavior, allowing accurate comparison across streams.

Flexible Sync Modes: Users can choose how synchronization is determined:
none: Pass-through without synchronization.
median: Uses the median timestamp across active streams as the reference.
index: Uses a user-specified stream-index as the reference stream.

Drop Policy: Buffers that are outside the synchronization window can be automatically dropped, and a GAP event is pushed downstream. This helps maintain visual consistency and avoid visual stutter.

Low-Latency Queuing: Internally, the plug-in uses GStreamer pads with a dedicated task for pushing synchronized buffers. The queuing logic minimises delay and supports min/max latency constraints.

Callback System: Drop, sync, flush, and failure events are handled by a callback mechanism. These are logged and reported via GStreamer debug output.

Properties

sync-mode: enumerator to select the synchronization strategy. Options are:
none: disables synchronization (default passthrough).
median: synchronizes buffers around the median `pts`.
index: synchronizes all buffers to match the timestamp of a master stream.

Example: sync-mode=median

stream-index: used in `index` mode. This unsigned integer identifies which stream acts as the master time reference (e.g., 0, 1, 2).

Example: stream-index=0

drop-rate: unsigned integer to control artificial dropping of buffers. When set to `N`, one out of every `N` buffers will be dropped intentionally. Useful for testing drop scenarios.

Example: drop-rate=10

min-latency: minimum latency (in nanoseconds) enforced by the internal queue before buffers are released. Ensures buffers are held long enough to allow synchronization.

Example: min-latency=10000000 (10 ms)

max-latency: maximum latency (in nanoseconds) after which a buffer is dropped if not synchronized. Prevents blocking on very late buffers.

Example: max-latency=30000000 (30 ms)

All properties are designed to be used at runtime or during pipeline construction. They provide flexibility to adapt the plug-in behavior to different use cases such as robotics, stereo vision, and multi-camera capture pipelines.

Missing Something?

Need to support another format, feature, or custom event mechanism? Let’s talk and extend the plug-in: Contact Us

Index

❯