GstRerunSink - Features and Limitations

From RidgeRun Developer Wiki



  Index Next: Building and Installation Guide




GstRerunSink general characteristics

GstRerunSink characteristics:

  • Easy Rerun integration into GStreamer pipelines.
  • Supported and tested on Linux devices with GStreamer-1.20 or newer.
  • Platform agnostic

Key Features

GstRerunSink key features:

The gst-rerunsink plugin provides the following capabilities:

  • Integration with Rerun: Seamlessly streams video frames into the Rerun visualization tool for real-time or offline data inspection.
  • Multiple Output Modes:
    • Spawn a local Rerun GUI viewer.
    • Stream to a remote Rerun viewer over gRPC.
    • Save frames to a .rrd file for post-processing and analysis.
  • Flexible Format Support:
    • Raw video formats: NV12, I420, RGB, RGBA, GRAY8.
    • Compressed formats: H.264 and H.265. Important: supported by the GStreamer element but not by Rerun. Currently GstRerunsink will accept H.264 and H.265 buffers but will not process or send the buffers to the rerun library.
    • NVIDIA-specific video/x-raw(memory:NVMM) for Jetson hardware acceleration.
  • Customizable Entity Paths: Assign logical paths like "camera/front/frame"' for organizing frame logs in Rerun.
  • Session Management:
    • Set unique recording-id values to distinguish between visualization sessions.
    • Throttle rendering rate and set a maximum bitrate to balance performance.
  • Synchronization Options:
    • Enable or disable synchronization with the GStreamer clock.
    • Adjust render-delay, ts-offset, and processing-deadline for precise control.
  • Quality-of-Service (QoS) Support: Emit upstream QoS events for adaptive streaming control.
  • Jetson Compatibility: Optimized for use on NVIDIA Jetson platforms, leveraging NVMM and zero-copy memory handling.

Current GstRerunSink inspect

gst-inspect-1.0 rerunsink
Factory Details:
  Rank                     none (0)
  Long-name                RerunSink
  Klass                    Sink/Video
  Description              Video sink that logs frames to Rerun for visualization
  Author                   Frander Diaz <frander.diaz@ridgerun.com>

Plugin Details:
  Name                     rerunsink
  Description              GStreamer sink plugin for Rerun visualization
  Filename                 ./libgstrerunsink.so
  Version                  1.0
  License                  LGPL
  Source module            rerunsink
  Binary package           GStreamer Rerun Sink
  Origin URL               https://github.com/rerun-io/rerun

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstRerunSink

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)NV12, (string)I420, (string)RGB, (string)GRAY8, (string)RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:NVMM)
                 format: { (string)NV12 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: video/x-h264
          stream-format: { (string)avc, (string)byte-stream }
      video/x-h265
          stream-format: { (string)hvc1, (string)hev1, (string)byte-stream }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  grpc-address        : gRPC server address (if non-default, connects via gRPC instead of spawning viewer)
                        flags: readable, writable
                        String. Default: "127.0.0.1:9876"
  image-path          : Entity path for logging images (e.g. 'camera/front/frame')
                        flags: readable, writable
                        String. Default: null
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 5000000 
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "rerunsink0"
  output-file         : Path to output .rrd file (if set, saves to disk instead of spawning viewer)
                        flags: readable, writable
                        String. Default: null
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  processing-deadline : Maximum processing deadline in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 15000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  recording-id        : Rerun recording/session identifier
                        flags: readable, writable
                        String. Default: null
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  spawn-viewer        : Spawn a Rerun viewer instance (ignored if output-file is set or grpc-address is non-default)
                        flags: readable, writable
                        Boolean. Default: true
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0

Limitations

The current Rerun library does not support H.264 or H.265. In the GstRerunSink element, the H.264 and H.265 buffers can be received, but no processing will be done by the Rerun library.



  Index Next: Building and Installation Guide