Jump to content

GStreamer Latency Benchmarks on Dragonwing EVK Board

From RidgeRun Developer Wiki


Follow us on: YouTube Twitter LinkedIn Email Share this page

Share This Page

Qualcomm Dragonwing Technical Guide logo
Qualcomm Dragonwing Technical Guide

Qualcomm Dragonwing Technical Guide
Table of Contents [Sticky]




On the Qualcomm Dragonwing IQ-9075 EVK, RidgeRun measured per-element GStreamer latency across software, VPU, GL/GPU-assisted, and camera-recording pipelines. The total pipeline latency is measured from the moment a buffer is pushed downstream by the source element until it reaches the sink element, using 300 samples of 1920x1080 video frames at 30 fps. In the documented tests, the VPU H.264 encoder v4l2h264enc measured about 3.9 ms average element latency and the VPU H.264 decoder v4l2h264dec about 1.6 ms, while the tested software H.264 encoding configuration showed dramatically higher latency. These numbers are pipeline-specific and must be interpreted with the exact caps, software versions, memory path, and test conditions below. A hardware accelerated video encoding pipeline takes in average 4.7 ms to deliver the encoded video image to the sink, while the same pipeline using a software codec takes 4105 ms in average. The gap between the latencies for hardware accelerated video decoding is smaller but still significant, a software pipeline delivers a decoded video frame with a latency of 7.7 ms in average, while a hardware accelerated pipeline takes 1.7 ms.


Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help .


Simple Pipeline

The following pipeline connects a simple synthetic source to fake sink. In average, buffers take 0.14 ms to travel from the source to the sink. 

 gst-launch-1.0 videotestsrc is-live=true num-buffers=300 ! "video/x-raw,width=1920,height=1080,framerate=30/1" ! videoconvert ! fakesink
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
videoconvert0 300 0.090 0.055 0.159 0.087 0.107 0.122 0.150
capsfilter0 300 0.051 0.026 0.089 0.049 0.062 0.069 0.078
Total Avg (ms) 0.140
Total Avg (ns) 140331

Software MP4 Encoding Pipeline

The following pipeline encodes synthetic video frames using the H.264 software codec. The software codec accounts for most of the total latency, making an average of 4105 ms. 

 gst-launch-1.0 -v videotestsrc is-live=true num-buffers=300 ! "video/x-raw,width=1920,height=1080,framerate=30/1" ! videoconvert ! x264enc ! mp4mux ! filesink location=test-video.mp4
ELEMENT COUNT Average
(ms)		 Minimum
(ms)		 Maximum
(ms)		 P50
(ms)		 P90
(ms)		 P95
(ms)		 P99
(ms)
x264enc0 291 4105.20 1977.19 6905.19 3936.793 5775.00 6340.63 6796.03
mp4mux0 300 0.549 0.098 4.186 0.226 0.873 3.201 3.625
videoconvert0 300 0.093 0.067 3.127 0.077 0.103 0.109 0.139
capsfilter0 300 0.049 0.033 0.086 0.048 0.056 0.063 0.075
Total Avg (ms) 4105.898
Total Avg (ns) 410589767

Software MP4 Decoding Pipeline

The following pipeline reads an MP4 video of 1920x1080@30 fps and decodes the video frames. In average, buffers takes 7.7 ms to reach the sink. 

gst-launch-1.0 -v filesrc location=test-video.mp4 ! qtdemux ! h264parse ! avdec_h264 ! fakesink
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
avdec_h264-0 151 7.589 2.435 11.529 8.184 10.059 10.468 10.990
h264parse0 300 0.096 0.038 0.317 0.088 0.133 0.144 0.176
qtdemux0 300 0.044 0.018 0.109 0.044 0.057 0.060 0.087
Total Avg (ms) 7.730
Total Avg (ns) 7729636

VPU accelerated MP4 Encoding Pipeline

The following pipeline encodes synthetic 1920x1080 video frames using the hardware accelerated H.264 codec and stores the video in a MP4 container. In average, buffers take 4.7 ms to reach the sink element. 

 gst-launch-1.0 -e -v videotestsrc is-live=true num-buffers=300 ! "video/x-raw,width=1920,height=1080,framerate=30/1" ! videoconvert ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=test-video-v4l2.mp4
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
v4l2h264enc0 300 3.920 3.119 10.432 3.923 4.311 4.331 4.598
mp4mux0 300 0.389 0.044 0.684 0.450 0.607 0.629 0.672
h264parse0 300 0.232 0.105 2.049 0.223 0.326 0.349 0.400
videoconvert0 300 0.111 0.056 0.208 0.109 0.120 0.143 0.175
capsfilter0 300 0.077 0.031 0.138 0.077 0.082 0.083 0.122
Total Avg (ms) 4.729
Total Avg (ns) 4728737

VPU accelerated MP4 Decoding Pipeline

The following pipeline read a MP4 video and decodes it using the hardware accelerated codec. In average, buffers take 1.7 ms to reach the sink element. 

gst-launch-1.0 -v filesrc location=test-video-v4l2.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! fakesink
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
v4l2h264dec0 300 1.595 0.654 8.088 1.564 2.082 2.210 2.545
h264parse0 300 0.085 0.049 0.448 0.073 0.123 0.165 0.237
qtdemux0 300 0.044 0.026 0.114 0.038 0.061 0.075 0.110
Total Avg (ms) 1.723
Total Avg (ns) 1723325

Hardware Accelerated Source and Display Pipeline

The following pipeline generates synthetic video as OpenGL textures on GPU, and renders the vido to display using OpenGL. 

 gst-launch-1.0 -v gltestsrc is-live=true num-buffers=300 ! "video/x-raw(memory:GLMemory),format=RGBA,width=1920,height=1080,framerate=30/1" ! glimagesink
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
gluploadelement0 300 0.103 0.083 0.223 0.099 0.125 0.135 0.177
capsfilter0 300 0.071 0.033 0.131 0.070 0.075 0.079 0.108
glcolorbalance0 300 0.046 0.022 0.108 0.036 0.083 0.088 0.100
glcolorconvertelement0 300 0.041 0.021 0.109 0.037 0.041 0.067 0.097
Total Avg (ms) 0.261
Total Avg (ns) 260705

Hardware Accelerated MP4 Encoding Pipeline

The following pipeline generates synthetic video as OpenGL textures on GPU, converts the image format from RGBA to NV12, downloads the image from GPU memory to RAM, encodes the video using the hardware accelerated H.264 codec and saves the video as a MP4 file. 

gst-launch-1.0 -v gltestsrc is-live=true num-buffers=300 ! "video/x-raw(memory:GLMemory),format=RGBA,width=1920,height=1080,framerate=30/1" ! glcolorconvert ! gldownload ! "video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1" ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=gltestsrc-hw.mp4
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
v4l2h264enc0 300 12.649 6.691 17.707 12.730 14.558 15.204 16.562
h264parse0 300 4.305 0.422 11.472 4.177 7.386 9.345 10.540
gldownloadelement0 300 2.513 0.335 13.888 2.767 3.043 3.148 3.330
glcolorconvertelement0 300 1.602 0.725 14.020 1.616 1.738 1.767 1.811
mp4mux0 300 0.469 0.057 0.682 0.565 0.595 0.607 0.629
capsfilter1 300 0.117 0.043 7.097 0.092 0.116 0.126 0.139
capsfilter0 300 0.080 0.030 0.141 0.081 0.096 0.098 0.111
Total Avg (ms) 21.734
Total Avg (ns) 21734135

Hardware Accelerated MP4 Decoding Pipeline

The following pipeline reads a MP4 video, decodes using the hardware codec, uploads the image as an OpenGL texture to GPU memory and renders it to display. 

gst-launch-1.0 -v filesrc location=gltestsrc-hw.mp4 ! qtdemux ! h264parse ! avdec_h264 ! glupload ! glimagesink
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
avdec_h264-0 300 240.334 8.167 259.325 243.318 248.689 250.206 254.286
gluploadelement0 300 1.622 0.136 3.513 1.336 3.287 3.326 3.396
glcolorconvertelement0 300 1.424 0.544 24.528 1.270 1.842 1.901 1.989
h264parse0 300 0.125 0.042 0.272 0.116 0.156 0.161 0.201
gluploadelement1 300 0.114 0.037 0.189 0.131 0.159 0.171 0.179
glcolorbalance0 300 0.102 0.044 0.183 0.120 0.150 0.155 0.166
qtdemux0 300 0.065 0.027 0.148 0.063 0.075 0.090 0.108
Total Avg (ms) 243.787
Total Avg (ns) 243786789

Camera recording at 1280x720@30fps

The following pipeline captures 1280x720@30 fps video from a MIPI camera, encodes it using the H.264 hardware codec and saves the video as MP4. In average, buffers take 3.7 ms to reach the sink. 

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! \ video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,\ interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc \ capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,\ video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-test.mp4
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
v4l2h264enc0 323 3.195 1.970 7.934 3.174 3.537 3.582 3.643
mp4mux0 323 0.261 0.062 1.508 0.171 0.588 0.614 0.715
h264parse0 323 0.195 0.124 0.689 0.198 0.258 0.271 0.311
capsfilter0 323 0.071 0.043 0.125 0.070 0.076 0.103 0.114
Total Avg (ms) 3.722
Total Avg (ns) 3721899

Camera recording at 1920x1080@30fps

The following pipeline captures 1920x1080@30 fps video from a MIPI camera, encodes it using the H.264 hardware codec and saves the video as MP4. In average, buffers take 5.3 ms to reach the sink. 

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! \ video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1,\ interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc \ capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,\ video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-test.mp4
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
v4l2h264enc0 300 4.691 3.028 11.355 4.718 5.031 5.105 5.174
mp4mux0 300 0.333 0.062 0.753 0.270 0.616 0.631 0.693
h264parse0 300 0.224 0.118 0.758 0.223 0.269 0.300 0.389
capsfilter0 300 0.069 0.043 0.181 0.068 0.077 0.100 0.120
Total Avg (ms) 5.318
Total Avg (ns) 5317801

Camera recording at 4K@30fps

The following pipeline captures 3840x2160@30 fps video from a MIPI camera, encodes it using the H.264 hardware codec and saves the video as MP4. In average, buffers take 10.8 ms to reach the sink. 

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! \ video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1,\ interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc \ capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,\ video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-test.mp4
ELEMENT COUNT Average (ms) Minimum (ms) Maximum (ms) P50 (ms) P90 (ms) P95 (ms) P99 (ms)
v4l2h264enc0 297 10.137 7.969 28.000 10.043 10.616 10.747 12.042
mp4mux0 297 0.362 0.058 1.257 0.362 0.633 0.688 0.847
h264parse0 297 0.224 0.119 0.994 0.208 0.283 0.313 0.631
capsfilter0 297 0.072 0.033 0.226 0.070 0.077 0.103 0.114
Total Avg (ms) 10.796
Total Avg (ns) 10796452

Key Takeaways

  • VPU-accelerated H.264 encoding with v4l2h264enc reports a 4.729 ms average pipeline latency for the synthetic 1080p30 test.
  • VPU-accelerated H.264 decoding with v4l2h264dec reports a 1.723 ms average pipeline latency for the tested MP4 input.
  • Camera recording latency increases with resolution: 3.722 ms at 720p30, 5.318 ms at 1080p30, and 10.796 ms at 4K30.
  • The OpenGL-to-H.264 path is slower than the simpler VPU encode path because it includes GPU color conversion, GPU-to-RAM download, parsing, muxing, and encoding.
  • These measurements are element/pipeline buffer-processing latency, not glass-to-glass latency. See Building Low-Latency Video Streaming Pipelines for end-to-end latency concepts.

Sources and Related Pages

FAQ

What is the GStreamer H.264 encoding latency on the Qualcomm Dragonwing IQ-9075 EVK?
In the 1920x1080 at 30 fps synthetic benchmark on this page, the `v4l2h264enc` VPU pipeline reports 4.729 ms average total latency. The software `x264enc` result is currently reported as 4105.898 ms.
What is the H.264 decoding latency with the Dragonwing VPU?
The tested `v4l2h264dec` pipeline reports 1.723 ms average total latency for the 1080p MP4 test.
How does camera recording latency change with resolution?
The measured average is 3.722 ms at 1280x720@30 fps, 5.318 ms at 1920x1080@30 fps, and 10.796 ms at 3840x2160@30 fps.
Are these glass-to-glass latency measurements?
No. These are GStreamer buffer-processing measurements within the tested pipelines. Glass-to-glass latency also includes sensor exposure, capture, queues, network transport when applicable, receiver buffering, display synchronization, and panel response.
How can I profile latency in my own GStreamer pipeline?
Use RidgeRun's GstShark tracing tools or the Pipeline Latency workflow linked from GStreamer Performance.

Qualcomm Dragonwing Technical Guide logo
Qualcomm Dragonwing Technical Guide
Back to top ↑


Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Qualcomm_Dragonwing_Technical_Guide/GStreamer/Performance/Element_Latency&oldid=86916"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.