DM365 LeopardBoard network video streaming latency test

From RidgeRun Developer Connection

(Difference between revisions)
Jump to:navigation, search
m (PC receiving video only pipeline)
m (Post power-on commands)
(3 intermediate revisions not shown)
Line 404: Line 404:
{| border=2
{| border=2
-
! Stopwatch<br>(sec) !! Live Video<br>(sec) !! Frame Latency<br>(frames, assuming 30 fps)
+
! Stopwatch<br>(sec) !! Live Video<br>(sec) !! Latency<br>(ms) !! Frame Latency<br>(frames, assuming 30 fps)
|-
|-
| 58.31 || 58.22 || 90 || 3
| 58.31 || 58.22 || 90 || 3
Line 420: Line 420:
| 01.22 || 01.14 || 80 || 3
| 01.22 || 01.14 || 80 || 3
|}
|}
 +
 +
=== Latency measurement errors ===
 +
 +
Using the on screen timer/clock approach can lead to measurement inaccuracies, including, but not limited to:
 +
 +
* monitor vertical refresh cycle
 +
* accuracy in the timer/clock being updated
 +
* delays in the updated timer/clock being display on the monitor
== Post power-on commands ==
== Post power-on commands ==
Line 429: Line 437:
Telnet to the target hardware and run top if you are interested in monitoring the ARM CPU load.
Telnet to the target hardware and run top if you are interested in monitoring the ARM CPU load.
 +
 +
= Optimization ideas =
 +
 +
Desktop example:
 +
 +
<pre>
 +
gst-launch rtspsrc location=rtsp://10.0.0.24:554/axis-media/media.amp latency=30 ! rtph264depay queue-delay=0 ! ffdec_h264 ! ffmpegcolorspace ! autovideosink
 +
</pre>
 +
 +
rtph264depay has a queue setting as well as rtspsrc.  Setting rtspsrc latency to zero the video frames are not very smooth.
 +
Increasing the value to around 30 causes the video is smooth and appears low-latency (not yet tested with camera / screenshot). udpsrc likely has a similar parameter.  Also the introducing queue  elements adds some latency (hopefully configurable).
 +
 +
 +
[[Category:DM36x]] [[Category:LeopardBoard]] [[Category:GStreamer]]
[[Category:DM36x]] [[Category:LeopardBoard]] [[Category:GStreamer]]

Revision as of 22:26, 30 October 2012

A simple test to get a feel for the amount of system latency when RTP network streaming live video from a LeopardBoard 365 to a PC. A website that displays a running stop watch (http://stopwatch.onlineclock.net) is captured by the target hardware camera, streamed over the network using RTP and displayed on the PC. The windows on the PC are arranged in such a manner that both the stop watch web page and the live video screen are visible. The PrintScreen feature is used to grab a snapshot of both screens at the same time. The difference between the time on the stopwatch and the time displayed on the live video screen shows the approximate system latency.

Contents

Bootloader, kernel, filesystem

The RidgeRun professional SDK using the DDOMPE branch for the hardware accelerated TI DMAI GStreamer plug-in.

Image loading instructions

Upgrading u-boot, using TFTP and DHCP.

setenv bootcmd
setenv autostart no
saveenv
setenv autoload no
dhcp
setenv serverip 10.111.0.4
tftp 0x82000000 upgrade_bootloader.bin.tfischer.leopard
go 0x82000000
setenv autoload no
dhcp
setenv serverip 10.111.0.4
setenv autostart no
tftp 0x81600000 ubl.nandbin.tfischer.leopard
tftp 0x81700000 bootloader.nandbin.tfischer.leopard
nand erase 0x20000 0x20000
nand erase 0x80000 0x3E0000
nand erase 0x3E0000 0x20000
nand write 0x81600000 0x20000 0x20000
nand write 0x81700000 0x80000 0x3E0000

Power cycle and interrupt autostart if needed before installing kernel.

Install kernel:

setenv autoload no
dhcp
setenv serverip 10.111.0.4
setenv autostart no
tftp 0x82000000 kernel.uImage.tfischer.leopard
setenv autostart yes
nand unlock 0x400000 0x440000
nand erase 0x400000 0x440000
nand write 0x82000000 0x400000 0x440000
setenv kernelloadaddr 0x82000000
setenv kernelsaveaddr 0x400000
setenv kerneloffset 0x400000
setenv ksize 0x440000
setenv kernelmd5sum 0bc189af1ca21d8a306ff30164e45398
saveenv

Setup u-boot environment:

setenv bootargs 'davinci_enc_mngr.ch0_output=COMPONENT davinci_enc_mngr.ch0_mode=720P-60 davinci_display.cont2_bufsize=6291456 vpfe_capture.cont_bufoffset=6291456 vpfe_capture.cont_bufsize=6291456 console=ttyS0,115200n8 video=davincifb:vid0=off:vid1=off:osd1=off dm365_imp.oper_mode=0 dm365_imp.oper_mode=0 vpfe_capture.interface=1 mem=54M root=/dev/nfs nfsroot=10.111.0.4:/local/home/tfischer/work/nict/fs/fs rw ip=dhcp mtdparts=davinci_nand.0:3968k(UBOOT),128k(UBOOT_ENV),4352k(KERNEL),204800k(FS)'
setenv bootcmd nboot \${kernelloadaddr} 0 \${kerneloffset}
setenv autostart yes
setenv autoload no
saveenv

U-boot parameter setup instructions

bootdelay=4
baudrate=115200
bootargs=davinci_enc_mngr.ch0_output=COMPONENT davinci_enc_mngr.ch0_mode=720P-60 davinci_display.cont2_bufsize=6291456 vpfe_capture.cont_bufoffset=6291456 vpfe_capture.cont_bufsize=6291456 console=ttyS0,115200n8 video=davincifb:vid0=off:vid1=off:osd1=off dm365_imp.oper_mode=0 davinci_capture.device_type=4 vpfe_capture.interface=1 mem=54M root=/dev/nfs nfsroot=10.111.0.4:/local/home/tfischer/work/leo365sdk/fs/fs rw ip=dhcp mtdparts=nand_davinci.0:3968k(UBOOT),128k(UBOOT_ENV),4352k(KERNEL),204800k(FS)
bootcmd=nboot ${kernelloadaddr} 0 ${kerneloffset}
autoload=no
bootfile=uImage
filesize=43B0C0
fileaddr=82000000
ipaddr=10.111.0.181
serverip=10.111.0.4
dnsip=10.111.0.1
autostart=yes
kernelloadaddr=0x82000000
kernelsaveaddr=0x400000
kerneloffset=0x400000
ksize=0x440000
kernelmd5sum=b9939ea8e60bd1780cb924e35d3c6596
stdin=serial
stdout=serial
stderr=serial
ethaddr=00:0C:0C:A0:02:b5
ver=U-Boot 1.3.4-svn9 (Sep  8 2010 - 08:09:02)

Output from successful boot

Using root NFS file system.

DM36x initialization passed!
UBL customized by RidgeRun
TI UBL Version: 1.50
Booting Catalog Boot Loader
BootMode = NAND
Starting NAND Copy...
Valid magicnum, 0xA1ACED66, found in block 0x00000004.
   DONE
Jumping to entry point at 0x81080000.


U-Boot 2010.06 (Mar 07 2011 - 16:21:03)

I2C:   ready
DRAM:  128 MiB
NAND:  256 MiB
Bad block table found at page 131008, version 0x01
Bad block table found at page 130944, version 0x01
Net:   Ethernet PHY: GENERIC @ 0x00
DaVinci EMAC
Hit any key to stop autoboot:  0 

Loading from nand0, offset 0x400000
   Image Name:   "RR Linux Kernel"
   Created:      2011-03-07  23:22:29 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    4411200 Bytes = 4.2 MiB
   Load Address: 80008000
   Entry Point:  80008000
Automatic boot of image at addr 0x82000000 ...
## Booting kernel from Legacy Image at 82000000 ...
   Image Name:   "RR Linux Kernel"
   Created:      2011-03-07  23:22:29 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    4411200 Bytes = 4.2 MiB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

Linux version 2.6.32-rc2-ridgerun (tfischer@contra-lx) (gcc version 4.3.3 (Sourcery G++ Lite 2009q1-203) ) #1 PREEMPT Mon Mar 7 15:48:51 MST 2011
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00053177
CPU: VIVT data cache, VIVT instruction cache
Machine: DM365 Leopard
Memory policy: ECC disabled, Data cache writeback
DaVinci dm365_rev1.2 variant 0x8
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 13716
Kernel command line: davinci_enc_mngr.ch0_output=COMPONENT davinci_enc_mngr.ch0_mode=720P-60 davinci_display.cont2_bufsize=6291456 vpfe_capture.cont_bufoffset=6291456 vpfe_capture.cont_bufsize=6291456 console=ttyS0,115200n8 video=davincifb:vid0=off:vid1=off:osd1=off dm365_imp.oper_mode=0 dm365_imp.oper_mode=0 vpfe_capture.interface=1 mem=54M root=/dev/nfs nfsroot=10.111.0.4:/local/home/tfischer/work/nict/fs/fs rw ip=dhcp mtdparts=davinci_nand.0:3968k(UBOOT),128k(UBOOT_ENV),4352k(KERNEL),204800k(FS)
PID hash table entries: 256 (order: -2, 1024 bytes)
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Memory: 54MB = 54MB total
Memory: 50132KB available (3964K code, 385K data, 144K init, 0K highmem)
SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Hierarchical RCU implementation.
NR_IRQS:245
Console: colour dummy device 80x30
Calibrating delay loop... 147.86 BogoMIPS (lpj=739328)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
DaVinci: 8 gpio irqs
NET: Registered protocol family 16
davinci_serial_init:97: failed to get UART2 clock
bio: create slab <bio-0> at 0
DM365 IPIPE initialized in Continuous mode
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
vpss vpss: dm365_vpss vpss probed
vpss vpss: dm365_vpss vpss probe success
dm365_afew_hw_init
ch0 default output "COMPONENT", mode "720P-60"
VPBE Encoder Initialized
cfg80211: Using static regulatory domain info
cfg80211: Regulatory domain: US
	(start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
	(2402000 KHz - 2472000 KHz @ 40000 KHz), (600 mBi, 2700 mBm)
	(5170000 KHz - 5190000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
	(5190000 KHz - 5210000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
	(5210000 KHz - 5230000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
	(5230000 KHz - 5330000 KHz @ 40000 KHz), (600 mBi, 2300 mBm)
	(5735000 KHz - 5835000 KHz @ 40000 KHz), (600 mBi, 3000 mBm)
cfg80211: Calling CRDA for country: US
LogicPD encoder initialized
Leopard DVI encoder initialized
Switching to clocksource timer0_1
musb_hdrc: version 6.0, cppi-dma, host, debug=0
musb_hdrc: USB Host mode controller at fec64000 using DMA, IRQ 12
musb_hdrc musb_hdrc: MUSB HDRC host driver
musb_hdrc musb_hdrc: new USB bus registered, assigned bus number 1
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP reno registered
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
JFFS2 version 2.2. (NAND) � 2001-2006 Red Hat, Inc.
msgmni has been set to 98
alg: No test for stdrng (krng)
io scheduler noop registered
io scheduler anticipatory registered (default)
davincifb davincifb.0: dm_osd0_fb: 1280x720x16@0,0 with framebuffer size 1800KB
davincifb davincifb.0: dm_osd1_fb: 1280x720x4@0,0 with framebuffer size 1800KB
DM365 IPIPEIF probed
imp serializer initialized
davinci_previewer initialized
davinci_resizer initialized
Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0x1c20000 (irq = 40) is a 16550A
console [ttyS0] enabled
brd: module loaded
NAND device: Manufacturer ID: 0x2c, Chip ID: 0xda (Micron NAND 256MiB 3,3V 8-bit)
4 cmdlinepart partitions found on MTD device davinci_nand.0
Creating 4 MTD partitions on "davinci_nand.0":
0x000000000000-0x0000003e0000 : "UBOOT"
0x0000003e0000-0x000000400000 : "UBOOT_ENV"
0x000000400000-0x000000840000 : "KERNEL"
0x000000840000-0x00000d040000 : "FS"
davinci_nand davinci_nand.0: controller rev. 2.3
console [netcon0] enabled
netconsole: network logging started
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usbcore: registered new interface driver usbtest
i2c /dev entries driver
Linux video capture interface: v2.00
ths7303 1-002c: chip found @ 0x58 (DaVinci I2C adapter)
vpfe_init
vpfe-capture: vpss clock vpss_master enabled
vpfe-capture vpfe-capture: v4l2 device registered
vpfe-capture vpfe-capture: video device registered
mt9p031 1-0048: Detected a MT9P031 chip ID 1801
mt9p031 1-0048: mt9p031 1-0048 decoder driver registered !!
vpfe-capture vpfe-capture: v4l2 sub device mt9p031 registered
vpfe_register_ccdc_device: DM365 ISIF
DM365 ISIF is registered with vpfe.
af major#: 252, minor# 0
AF Driver initialized
aew major#: 251, minor# 0
AEW Driver initialized
Trying to register davinci display video device.
layer=c23dc600,layer->video_dev=c23dc770
Trying to register davinci display video device.
layer=c23dc200,layer->video_dev=c23dc370
davinci_init:DaVinci V4L2 Display Driver V1.0 loaded
watchdog watchdog: heartbeat 60 sec
davinci_mmc davinci_mmc.0: Using DMA, 4-bit mode
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
Advanced Linux Sound Architecture Driver Version 1.0.21.
No device for DAI tlv320aic3x
No device for DAI davinci-i2s
asoc: tlv320aic3x <-> davinci-i2s mapping ok
ALSA device list:
  #0: DaVinci EVM (tlv320aic3x)
TCP cubic registered
NET: Registered protocol family 17
lib80211: common routines for IEEE802.11 drivers
Clocks: disable unused mmcsd1
Clocks: disable unused spi0
Clocks: disable unused spi1
Clocks: disable unused spi2
Clocks: disable unused spi3
Clocks: disable unused spi4
Clocks: disable unused pwm0
Clocks: disable unused pwm1
Clocks: disable unused pwm2
Clocks: disable unused pwm3
Clocks: disable unused timer1
Clocks: disable unused timer3
Clocks: disable unused emac
Clocks: disable unused voice_codec
Clocks: disable unused rto
Clocks: disable unused mjcp
davinci_emac_probe: using random MAC addr: b2:fc:37:9a:12:dd
emac-mii: probed
Sending DHCP requests ., OK
IP-Config: Got DHCP answer from 10.111.0.1, my address is 10.111.0.154
IP-Config: Complete:
     device=eth0, addr=10.111.0.154, mask=255.255.255.0, gw=10.111.0.1,
     host=10.111.0.154, domain=fischerfamily.org, nis-domain=fischerfamily.org,
     bootserver=10.111.0.1, rootserver=10.111.0.4, rootpath=
Looking up port of RPC 100003/2 on 10.111.0.4
Looking up port of RPC 100005/1 on 10.111.0.4
VFS: Mounted root (nfs filesystem) on device 0:14.
Freeing init memory: 144K
init started: BusyBox v1.18.2 (2011-03-07 15:51:49 MST)
starting pid 983, tty '': '/etc/rcS'
Starting System
done.
Welcome to
__________ .__     .___               __________                
\______   \|__|  __| _/  ____    ____ \______   \ __ __   ____  
 |       _/|  | / __ |  / ___\ _/ __ \ |       _/|  |  \ /    \ 
 |    |   \|  |/ /_/ | / /_/  >\  ___/ |    |   \|  |  /|   |  \
 |____|_  /|__|\____ | \___  /  \___  >|____|_  /|____/ |___|  /
        \/          \//_____/       \/        \/             \/ 
	   
    Embedded Linux Solutions
 
For further information see:
http://www.ridgerun.com
Build host: contra-lx
Built by: admin
Build date: Mon, 07 Mar 2011 16:20:50 -0700
Build tag: leopard
Configuring network interfaces
udhcpc (v1.18.2) started
Sending discover...
Sending select for 10.111.0.154...
Lease of 10.111.0.154 obtained, lease time 43200
Root filesystem is mounted over eth0 network interface, discarting IP obtained by userspace dhcp
deleting routers
adding dns 10.111.0.1

Please press Enter to activate this console. 
starting pid 1103, tty '/dev/ttyS0': '-/bin/sh'


BusyBox v1.18.2 (2011-03-07 15:51:49 MST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

/ # 

Loaded modules

The RidgeRun SDK uses a CMEM pool configuration designed for vide decode. The following pool values are used to work properly with video encode.

POOLS=1x524288,1x81920,2x8192,6x4096,8x96,64x56,1x2938880,4x675840,1x75840,4x1548288,1x33077952,5x1253376,3x4147200
modprobe cmemk phys_start=0x84200000 phys_end=0x88000000 pools=$POOLS
modprobe edmak.ko
modprobe irqk
modprobe dm365mmap.ko


fbset -disable
ipipe_init -d

GStreamer pipelines

Video only pipelines

DM365 sending video only pipeline

export HOST_IP=10.111.0.4

gst-launch -v -e \
   v4l2src always-copy=false ! \
   video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! \
   dmaiaccel ! \
   dmaiperf ! \
   queue ! \
   dmaienc_h264 targetbitrate=800000 encodingpreset=2 ! \
   rtph264pay ! \
   udpsink host=$HOST_IP port=10000

PC receiving video only pipeline

Before starting PC video playback, you may have to adjust the caps parameters to udpsrc in the gst-launch command below. Look around 40 lines down from the output of the target hardware gst-launch video transmit command for a line similar to:

/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, profile-level-id=(string)640028, sprop-parameter-sets=(string)\"J2QAKK2IDkOYIOEMKQpEByHMEHCGFIUiA5DmCDhDCkKQwEIYwhxmMhCGAhDGEOMxkIQwEIYwhxmMhCICEZjOI8KfEfiP4j8R8R4ziMREQoEIjEcR4j5PxH8n5PiPEcRkiLQCgC3I\\,KO48sA\\=\\=\", payload=(int)96, ssrc=(guint)3192754172, clock-base=(guint)71598694, seqnum-base=(guint)35200

Now adjust the following line to use the caps found in the target hardware gst-launch video transmit command.


CAPS="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, profile-level-id=(string)640028, sprop-parameter-sets=(string)\"J2QAKK2IDkOYIOEMKQpEByHMEHCGFIUiA5DmCDhDCkKQwEIYwhxmMhCGAhDGEOMxkIQwEIYwhxmMhCICEZjOI8KfEfiP4j8R8R4ziMREQoEIjEcR4j5PxH8n5PiPEcRkiLQCgC3I\\,KO48sA\\=\\=\", payload=(int)96, ssrc=(guint)3192754172, clock-base=(guint)71598694, seqnum-base=(guint)35200"

gst-launch -v \
   udpsrc port=10000 caps="$CAPS" ! \
   rtph264depay ! \
   ffdec_h264 ! \
   ffmpegcolorspace ! \
   queue ! \
   ximagesink

RidgeRun PrintScreen based latency results

Some of the samples were capture part way through a screen refresh, making the values hard to read, so those samples were discarded.

Example PrintScreen (the video capture driver was configured for LI-LBCM5M1 and I was using a LI-5M03 so it produced a mirror image).

Video-only-1.jpg

Captured latency:

Stopwatch
(sec)
Live Video
(sec)
Latency
(ms)
Frame Latency
(frames, assuming 30 fps)
58.31 58.22 90 3
34.68 34.32 360 11
51.91 51.85 60 2
04.91 04.74 170 6
23.62 23.45 170 6
39.39 39.13 260 8
01.22 01.14 80 3

Latency measurement errors

Using the on screen timer/clock approach can lead to measurement inaccuracies, including, but not limited to:

Post power-on commands

ifconfig eth0 # learn address to use to telnet to target hardware
inetd

Telnet to the target hardware and run top if you are interested in monitoring the ARM CPU load.

Optimization ideas

Desktop example:

gst-launch rtspsrc location=rtsp://10.0.0.24:554/axis-media/media.amp latency=30 ! rtph264depay queue-delay=0 ! ffdec_h264 ! ffmpegcolorspace ! autovideosink

rtph264depay has a queue setting as well as rtspsrc. Setting rtspsrc latency to zero the video frames are not very smooth. Increasing the value to around 30 causes the video is smooth and appears low-latency (not yet tested with camera / screenshot). udpsrc likely has a similar parameter. Also the introducing queue elements adds some latency (hopefully configurable).

Navigation
Toolbox