IMX6 Live Preview Latency Measurements

From RidgeRun Developer Wiki


Problems running the pipelines shown on this page?
Please see our GStreamer Debugging guide for help.

Introduction

This page shows the latency results with Gstreamer 1.0 pipelines running on an iMX6 SoC. The test's objective is to know the latency of the GPU on the iMX6 Variscite board using the iMX6 plugins and OpenGL plugin developed by RidgeRun.

The test environment is as the following:

Table 1. Development environment
Property Value
Camera ov5640_mipi
Resolution 1280x720
Frame rate 30 fps
Platform iMX6 Variscite

Test Setup

The testing was done using a iMX6 Variscite Rev 1.22, along with a video source using the OV5640_MIPI camera sensor board.

1. Run the Gstreamer pipeline on the target board. See each individual test for a specific pipeline.

2. Aim the camera to a huge stopwatch on the PC display.

3. Take a snapshot of LCD display and stopwatch while running.

4. If Source is the time displayed in the stopwatch and System is the time displayed by the host image, then the overall latency of the system is given by

Latency = Source - System

Test Results

Capture + Display

IPU

  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! imxipuvideosink -v

Latency: 43 ms

GPU

  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! imxg2dvideosink
  • Latency: 44 ms
  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! imxeglvivsink -v
  • Latency: 44 ms
  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! glessink -v
  • Latency: 44 ms

Capture + Transform + Display

GPU

  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30  ! imxg2dvideotransform ! "video/x-raw,\
format=(string)RGBA, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive" \
! imxg2dvideosink
  • Latency: 44 ms

Capture + Misosample + Display

IPU

Plugin: imxipuvideosink

  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! tee name=t ! queue ! misosample name=miso silent=true \
! 'video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, framerate=(fraction)30/1' \
! imxipuvideosink t. ! queue ! imxipuvideotransform ! 'video/x-raw,width=128,height=96' ! miso.sink_1
  • Latency: 130 ms
  • Pipeline (*)
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! tee name=t ! queue ! misosample name=miso silent=true \
! 'video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, framerate=(fraction)30/1' \
! imxipuvideosink t. ! queue ! miso.sink_1
  • Latency: 87 ms

GPU

Plugin: glessink

  • Pipeline (*)
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! tee name=t ! queue ! misosample name=miso silent=true \
! 'video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! glessink t. \
! queue ! miso.sink_1
  • Latency: 87 ms

Plugin: imxg2dvideosink

  • Pipeline (*)
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! tee name=t ! queue ! misosample name=miso silent=true \
! 'video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! imxg2dvideosink t. \
! queue ! miso.sink_1
  • Latency: 86 ms
  • Pipeline
gst-launch-1.0 imxv4l2src capture-mode=4 fps-n=30 ! tee name=t ! queue ! misosample name=miso silent=true \
! 'video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, framerate=(fraction)30/1' ! imxg2dvideosink t. \
! queue ! imxipuvideotransform ! 'video/x-raw,width=128,height=96'  miso.sink_1
  • Latency: 86 ms


Capture + OpenGL ES 2.0 + Display

This part specifies the results obtained after the modifications to the OpenGL algorithms implemented in the glessink plugin.

Render to texture

This implementation develops an algorithm for render to texture firstly, after render to screen form the texture rendered.

  • Pipeline
gst-launch-1.0 imxv4l2src capture_mode=4 fps-n=30 ! imxg2dvideotransform \
! 'video/x-raw,format=(string)RGBA, width=(int)1280, height=(int)720, framerate=(fraction)30/1' \
! glessink x-offset=0 y-offset=0
  • Latency: 44 ms

Note: Used the videotransform because it was necessary to convert of the camera capture format to the RGB.


(*) These pipelines do not include imxiputransform