GStreamer Buffer Synchronization: Basics and foundation - Introduction

GStreamer Buffer Synchronization
Basics and foundation
Introduction Applications
Getting Started
Evaluating Getting the Code Building
Documentation
API Reference GStreamer
Examples
API Reference GStreamer
Contact Us

Sensor Synchronization Foundations

There are multiple ways to synchronize sensors using hardware, software, or hybrid techniques. The GStreamer Buffer Synchronization plug-in uses software-based synchronization, where the incoming buffers are matched through their timestamps. This is particularly useful when the hardware is unavailable, resulting in a more affordable alternative.

This wiki covers the introduction to the synchronization technique used in this plug-in and the concepts required to understand what's under the hood.

Requirements for Software Sensor Synchronization

Synchronization at a software level implies having a common timestamp system (clock system), where all the sensors work with the same time reference. This time reference can be a TSC hardware chip or the Linux Monotonic Clock. Ideally, the clock should be monotonic, without room for dynamic or runtime adjustments, such as NTP. Therefore, a brief list of requirements is:

• The clock system shall be monotonic.
• The clock system cannot be modified at runtime.
• All sensors involved shall have the same clock system.

Synchronization Mechanism

Assuming the following scenario:

Frames coming from three streams into a stitcher application

In this case, the illustration shows that the last stream started at the time 66, whereas the first two streams began late, after the time 96. The time difference is more than 30 ms delay, where discrepancies can be spotted.

Concept of quantum. An ideal synchronization mechanism fits the buffers' timestamps within the quantum.

Synchronization with software timestamps can be achieved by placing all the buffers of a group to fit in a time slot (as illustrated in the figure above). The length of the time slot corresponds to the frame period (1/framerate).

Thus, the synchronization of the aforementioned case can be achieved by dropping the buffer 0 of the last stream, allowing the buffer 1 of this same stream to arrive at the stitcher, placing all buffers within the same quantum, as illustrated below.

Synchronization in different scenarios

Known limitations

The approach followed by this method has a limitation, specifically a time offset that corresponds to the frame period (+/- period/2). For instance, for a 60 fps stream, there is a time deviation of +/- 8.8 ms.