GstInterpipe: Difference between revisions

From RidgeRun Developer Wiki
No edit summary
mNo edit summary
 
(61 intermediate revisions by 5 users not shown)
Line 1: Line 1:
= Introduction =
{{GstInterpipe/Head| | next=GstInterpipe Overview|metakeywords=}}


== Overview ==
GstInterpipe is a Gstreamer plug-in that allows communication between two independent pipelines. The plug-in consists of two elements:
* interpipesink
* interpipesrc


Generally speaking, the idea is that given a source pipeline:
The following table of contents offers all you need to know about GstInterpipe project.
<pre style="background:#d6e4f1">
v4l2src ! interpipesink name=video_src
</pre>


Various sink pipelines may listen to it by:
<pre style="background:#d6e4f1">
interpipesrc listen-to=video_src ! xvimagesink
interpipesrc listen-to-video_src num-buffers=1 ! jpegenc ! filesink location=snapshot.jpeg
</pre>
The state of each pipeline is independent, each one can manage events by their own and can be attached or detached at any moment.
The concept behind the Interpipes Project is to simplify the construction of GStreamer applications, which often has the complexity of requiring dynamic pipelines. It transforms the construction process from low level pad probe manipulation to the higher level setting an element's parameter value.  Application developers don't get mired down in stalled pipelines because one branch of a complex pipeline changed state.
For example, take a look in a complex pipeline like the one in the figure 1:
[[File:Complex_pipeline.png|thumb|center|800px|Figure 1: Complex Pipeline]]
The complex pipeline of figure 1 can be constructed into smaller, independent pipelines using interpipe elements as it is illustrated in the figure 2:
[[File:Interpipe_pipeline.png|thumb|center|800px|Figure 2: Pipeline using Interpipes]]
This way the stream flow in a complex pipeline is reduced to simply set the correct listeners in the interpipe elements taking away the complexity of re-configuring pads or some other complex and error-prone logic.
== Features and Limitations ==
The GstInterpipe project exposes the following characteristics:
* Inspired by intervideosrc/intervideosink
* Uses appsrc and appsink as a base
* Generic data handling
* Multiple dynamic interpipesink - interpipesrc connections
* Replace tee and selector elements allowing dynamic pipeline flow control with GStreamer Daemon
The principal features the plug-in exposes are the following. These ones are explained with further detail later.
* '''Buffer transfer''' <br>Interpipesink will transfer automagically the buffers it receives to all the interpipesrcs that are currently listening to it.
*'''Dynamic switching''' <br> Interpipesrc can switch the interpipesink they are listening to at any time just by setting the property to the new value. No need to worry about the pipeline's state, dangerous events like EOS, pad probes, valves, selectors, pad links, etc...
*'''Caps negotiation''' <br> Even though interpipes break a big pipeline into smaller ones, the caps negotiation process takes into account all of them. This means that it is guaranteed that the negotiated caps will be supported by the source and all its listeners (or fail due to missing valid intersection).
*'''Event forwarding''' <br> Similar to buffers, events may be forwarded from the interpipesinks to the interpipesrcs, and viceversa. The project takes into account downstream and upstream events, as well as in-bounds and out-of-bands events.
*'''Timestamp synchronization''' <br> The base times of independent pipelines will likely be different. Given that a buffer will be transferred from one pipe to another, this may represent a problem in situations where synchronization is a must. GstInterpipe takes care of this situation by compensating the buffer's timestamps according to the pipeline's base time, ensuring appropriate synchronization.
*'''New node notification''' <br> An interpipesrc may be set to listen to an inexistent node-name. If this is the case, the interpipesrc will be registered to receive a notification when the desired interpipesink is created. At this point, the connection will be made and the buffer flow will start.
The current release exposes the following limitations and known bugs:
*'''Specialized clocks''' <br>At the time being, pipelines negotiate their clock independently. If a pipeline uses a special clock (i.e.: GstAudioSinkClock), the associated pipes (being independent) may negotiate different clocks, typically GstSystemClock. This could be a big problem if, for example, synchronization between streams is required. It is currently responsibility of the application to set the special clock in all the involved pipes by calling [https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstPipeline.html#gst-pipeline-set-clock gst_pipeline_set_clock].
= Getting the Software =
GstInterpipe is an open source project and can be downloaded from GitHub. The project is hosted at:
TODO
As with every open source project, there's an open invitation to provide feedback to the maintainers as bug reports, feature requests and even source code collaborations. The following table summarizes the recommended methods to collaborate with RidgeRun:
{| class="wikitable"
|-
! Collaboration !! Method
|-
| Bug report || GitHub's issue tracker
|-
| Feature request || GitHub's issue tracker
|-
| Patch Submission || Pull request
|}
= Installation Guide=
== Dependencies==
The following packages are needed in order to build and use gst-interpipe:
* gstreamer-1.0
* gstreamer-plugins-base-1.0
* gstreamer-app-1.0
* gstreamer-check-1.0
These are likely already installed in your OS distribution. In case you want to double check and install the missing packages, run the following commands according to your OS:
=== Debian Based ===
This includes Ubuntu, Kubuntu, Debian, etc...
sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
=== Mac OSX ===
Using Mac Ports:
sudo port install gstreamer1 gstreamer1-gst-plugins-base
== Building the project ==
In order to build the project run the following commands. Note that the libdir may vary according to your system.
./autogen.sh --libdir /usr/lib/x86_64-linux-gnu/gstreamer-1.0/
make
The ''autogen.sh'' script will automatically run the ''configure'' script. In case a more complex configuration is needed, the configure step may be executed manually:
./autogen.sh --noconfigure
./configure --libdir /usr/lib/x86_64-linux-gnu/gstreamer-1.0/ <additional advanced options>
make
Finally, the status of the current version may be checked by running the unit tests:
make check
== Installing the plugin ==
The plugin is installed to the GStreamer's default plug-in location in the file system by running:
sudo make install
If you don't want to install the plugin in the default location, it may also be found by setting GST_PLUGIN_PATH to the library location. For example:
GST_PLUGIN_PATH=/home/mgruner/gst-interpipe-1.0/gst/interpipe/.libs gst-inspect-1.0 interpipe
=User Guide=
The following sections describe in detail the different capabilities of the elements, configurations and examples.
== Features ==
=== Buffer Forwarding ===
The main purpose of the GstInterpipe Project is to communicate buffers from one pipeline to another. This can be done in a safe way, without worrying about the states or events of any of the pipelines. To set an interpipesrc (listener) to listen to a specific interpipesink (node), all it is needed is to set the ''listen-to'' property. A NULL or empty ''listen-to'' will disconnect the listener from any node. The following table summarises the properties involved in buffer transfer.
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesrc
|-
| '''Name''' || colspan="2" | listen-to
|-
| '''Description''' || colspan="2" | The name of the node to listen to
|-
| rowspan="2" | '''Special Values''' || NULL || Stop listening
|-
|  (empty) || Stop listening
|}
<br>
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesink
|-
| '''Name''' || colspan="2" | name
|-
| '''Description''' || colspan="2" | The name of the node. It can only be set during construction.
|-
| '''Special Values''' || colspan="2"|
|}
The following figures, and their respective gst-launch pipelines illustrate this concept:
gst-launch-1.0 \
videotestsrc ! interpipesink name=camera \
interpipesrc listen-to="" ! fakesink
[[File:Buffer_transfer_not_connected.png|600px|framed|center|Detached pipelines]]
gst-launch-1.0 \
videotestsrc ! interpipesink name=camera \
interpipesrc listen-to="camera" ! fakesink
[[File:Buffer_transfer_connected.png|600px|framed|center|Attached pipelines]]
=== Dynamic Switching ===
One of the greatest strengths of GstInterpipe is the ability of an interpipesrc to switch between different interpipesinks at runtime. To do so, change the ''listen-to'' property. Similarly, disconnections and reconnections can be performed by clearing and setting the property accordingly. When performing dynamic switching, caps must be taken into consideration. Specifically, a switch can be performed to a node in two scenarios:
;No other listeners
:Caps are re-negotiated between the two pipelines, looking for the optimum caps.
;Existing listeners
:The sink pipeline must support the caps already configured by the node and listeners, otherwise it'll fail.
The following figure illustrates this concept:
[[File:Dynamic_switch.png|600px|framed|center|Dynamic Switch Scenarios]]
The caps re-negotiation capability can be disabled with the ''allow-renegotiation'' property. The following table summarises the involved property:
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesrc
|-
| '''Name''' || colspan="2" | allow-renegotiation
|-
| '''Description''' || colspan="2" | Allow the interpipesrc to renegotiate caps when attached to an interpipesink with different caps.
|-
| '''Special Values''' || colspan="2"|
|}
The switch can be disabled entirely by setting the ''block-switch'' property.
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesrc
|-
| '''Name''' || colspan="2" | block-switch
|-
| '''Description''' || colspan="2" | Allow changing the node a listener is currently listening to.
|-
| '''Special Values''' || colspan="2"|
|}
Programatically, the dynamic switch can be performed as the following:
<pre>
/* Create pipelines */
GstElement *pipe1 = gst_parse_launch ("videotestsrc ! interpipesink name=camera1", NULL);
GstElement *pipe2 = gst_parse_launch ("videotestsrc ! interpipesink name=camera2", NULL);
GstElement *pipe3 = gst_parse_launch ("interpipesrc name=src listen-to=camera1 ! fakesink", NULL);
/* Grab a reference to the interpipesrc */
GstElement *src = gst_bin_get_by_name(pipe3, "src");
/* Perform the switch */
g_object_set (src, "listen-to", "camera2", NULL);
</pre>
=== Caps Negotiation ===
GstInterpipe takes into account the node and all its listeners during the caps negotiation process. It is guaranteed that the resulting supported set of caps will be an intersection between all the listeners and node caps. This only applies to the listeners that are connected during the caps negotiation process. If a listener is adhered later, it will be handled independently (see [[#Dynamic_Switching|Dynamic Switching]]).
The following figure shows this concept.
[[File:Caps_negotiation.png|600px|framed|center|Caps negotiation process]]
=== Event Forwarding ===
As with buffers, events can be transferred from the nodes to the listeners (and viceversa). GStreamer handles three types of events:
* '''Upstream Events''': events that travel contrary to the buffer flow
* '''Downstream Out-Of-Bounds Events''': events traveling in the same direction as the buffer flow, but sent immediately.
* '''Downstream In-Bounds Events''': events traveling in the same direction as the buffer flow, but sent serialised with the buffers.
GstInterpipe supports the three types of events, each one under different considerations. Additionally, the EOS is treated with special care, hence controlled by independent properties. The following sections describe these scenarios
==== Upstream Events ====
Upstream events are sent from the listeners to the node. Upstream events are only transferred if the event emitter is the only listener connected to the node. This was designed this way in order to avoid changing the node state to the other listeners. The following image describes this situations:
[[File:Events_upstream.png|600px|framed|center|Upstream events transfer]]
==== Downstream Out-Of-Bounds Events ====
Out-Of-Bounds downstream events are sent from the node to the listeners. These events are sent to all of the listeners as soon as they are received. By setting the ''forward-events'', the node can be configured to avoid sending events. By setting the ''accept-events'', individual listeners can be configured to avoid receiving events. The following tables summarise these properties:
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesink
|-
| '''Name''' || colspan="2" | forward-events
|-
| '''Description''' || colspan="2" | Allow the interpipesink to forward any kind of events to the listeners
|-
| '''Special Values''' || colspan="2"|
|}
<br>
<br>
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesrc
|-
| '''Name''' || colspan="2" | accept-events
|-
| '''Description''' || colspan="2" | Accept events coming from the node
|-
| '''Special Values''' || colspan="2"|
|}
Graphically, the following figure shows different configuration examples.
[[File:Events_downstream_oob.png|600px|framed|center|Downstream out-of-bounds event transfer]]
==== Downstream In-Bounds Events ====
In-bound downstream events travel from the node to all the listeners. The main difference is that they travel inlined with the buffer flow. Internally, both the interpipesink and interpipesrc con queue buffers, so serial events are queued as well so that they are pushed at the precise moment they need to be sent. The same set of properties control the transmission of these events.
Graphically, this can be appreciated as the following image:
[[File:Events_downstream_ib.png|600px|framed|center|Downstream in bounds event transfer]]
==== End-Of-Stream Events ====
EOS events are a special type of in-bounds downstream events. The same queuing principles apply to these. However, this event typically causes drastic pipeline consequences, to it is likely that a pipeline could want to receive all event types, but the EOS. For this reason, the EOS transfer
can be controlled by setting ''forward-eos'' to disable EOS transfer completely from the node, or ''accept-eos'' to selectively block the EOS in individual listeners. The following table summarises the properties.
Graphically, this can be appreciated as the following image:
[[File:Events_downstream_ib.png|600px|framed|center|Downstream in bounds event transfer]]
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesink
|-
| '''Name''' || colspan="2" | forward-eos
|-
| '''Description''' || colspan="2" | Allow the interpipesink to forward EOS events to the listeners
|-
| '''Special Values''' || colspan="2"|
|}
<br>
<br>
{| class="wikitable" style="margin:auto;"
<table>
|-
<tr>
! colspan="3" | Property
<td><div class="clear; float:right">__TOC__</div></td>
|-
<td>
| '''Element''' || colspan="2" | interpipesrc
{{Sponsor Button}}
|-
<td>
| '''Name''' || colspan="2" | accept-eos
</tr>
|-
</table>
| '''Description''' || colspan="2" | Accept EOS events coming from the node
|-
| '''Special Values''' || colspan="2"|
|}
 
=== Timestamp Synchronization ===
 
The transferred buffers will go through two (or more) different pipelines. Each pipeline will have its own base time and, hence, the buffer timestamp will have different meaning in each one. This can induce problems in pipelines where synchronisation is required. To solve this problem, the buffer timestamp is compensated taking into account the combination of the base times. The result is a equivalent relative buffer time. This feature, however, can be disabled if desired by setting the ''enable-sync'' property. The following table shows the property:
 
{| class="wikitable" style="margin:auto;"
|-
! colspan="3" | Property
|-
| '''Element''' || colspan="2" | interpipesrc
|-
| '''Name''' || colspan="2" | enable-sync
|-
| '''Description''' || colspan="2" | Compensate buffer timestamps in order to achieve equivalent buffer times
|-
| '''Special Values''' || colspan="2"|
|}


The following image exemplifies the synchronisation potential problems.
==Quick Overview==
GstInterpipe is a RidgeRun open source [https://www.ridgerun.com/gstreamer GStreamer] plug-in that enables pipeline buffers and events to flow between two or more independent pipelines. It consists of two elements: interpipesink and interpipesrc. The interpipesrc connects with an interpipesink, from which it receives buffers and events.


[[File:Timestamp_synchronization.png|600px|framed|center|Timestamp synchronization]]
GstInterpipe is intended to reduce a big complex pipeline system into smaller and simpler independent pipelines. You can view and handle the independent pipelines as different blocks that can be controlled independently.  Each pipeline change state, have element properties modified, and can connect or disconnect with other pipelines at runtime. GstInterpipe allows you to have multiple source and sink pipelines in an application.  The connections between source pipelines and sink pipelines can dynamically change at runtime. Is like if you have a set of different pipes that you can plug and unplug between them at runtime, so the system changes its behaviour according to the pipe interconnection configuration.


=== New Node Notification ===
==Promo/Demo Video==


A listener can be set to listen an inexistent node. This doesn't necessarily means an error. It is very common that the node could be created later. For this reason, internally the interpipe project uses a notification mechanism to notify the listeners when a new node is connected. If the listener is currently configured to listen to the new node, the connection will be stablished.
'''GStreamer Conference-2017 talk from RidgeRun''' : '''[[GStreamer_Daemon | GStreamer Daemon]] - building a media server in under 30 minutes'''


== Properties ==
*[https://vimeo.com/240547523 vimeo version] (may have better lipsync)
*embedvideo (below)
<center>
<embedvideo service="vimeo">https://vimeo.com/240547523</embedvideo>
</center>


TODO
== Getting Started ==
Start navigating this wiki by going to the [[GstInterpipe - GstInterpipe Overview|GstInterpipe Overview]] page in the table of contents.


= Examples =
{{ContactUs}}


TODO
{{GstInterpipe/Foot | | next= GstInterpipe Overview}}

Latest revision as of 19:29, 25 January 2024




  Index Next: GstInterpipe Overview


GstInterpipe
GstInterpipe Overview
Features and Limitations
Building and Installation Guide
Features Detailed Description
Developers Guide
GstInterpipes Elements Detailed Description
Examples
Contact Us




The following table of contents offers all you need to know about GstInterpipe project.



Quick Overview

GstInterpipe is a RidgeRun open source GStreamer plug-in that enables pipeline buffers and events to flow between two or more independent pipelines. It consists of two elements: interpipesink and interpipesrc. The interpipesrc connects with an interpipesink, from which it receives buffers and events.

GstInterpipe is intended to reduce a big complex pipeline system into smaller and simpler independent pipelines. You can view and handle the independent pipelines as different blocks that can be controlled independently. Each pipeline change state, have element properties modified, and can connect or disconnect with other pipelines at runtime. GstInterpipe allows you to have multiple source and sink pipelines in an application. The connections between source pipelines and sink pipelines can dynamically change at runtime. Is like if you have a set of different pipes that you can plug and unplug between them at runtime, so the system changes its behaviour according to the pipe interconnection configuration.

Promo/Demo Video

GStreamer Conference-2017 talk from RidgeRun : GStreamer Daemon - building a media server in under 30 minutes

Load video
Vimeo

Getting Started

Start navigating this wiki by going to the GstInterpipe Overview page in the table of contents.


RidgeRun Resources

Quick Start Client Engagement Process RidgeRun Blog Homepage
Technical and Sales Support RidgeRun Online Store RidgeRun Videos Contact Us
RidgeRun.ai: Artificial Intelligence | Generative AI | Machine Learning

Contact Us

Visit our Main Website for the RidgeRun Products and Online Store. RidgeRun Engineering information is available at RidgeRun Engineering Services, RidgeRun Professional Services, RidgeRun Subscription Model and Client Engagement Process wiki pages. Please email to support@ridgerun.com for technical questions and contactus@ridgerun.com for other queries. Contact details for sponsoring the RidgeRun GStreamer projects are available in Sponsor Projects page.


  Index Next: GstInterpipe Overview


Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=GstInterpipe&oldid=52624"