Video Capture for the Dragonwing EVK Board

Q: Can I use Yavta, libcam or v4l2 to capture from the IMX577?

No, to capture from the IMX577, qtiqmmfsrc or another application compatible with Qualcomm's camera firmware stack must be used.

Qualcomm Dragonwing Technical Guide

Table of Contents [Sticky]

This page provides the steps to enable the IMX577 camera capture on the Dragonwing IQ-9075 EVK. Camera capture is achieved using the qtiqmmfsrc, other frameworks like libcamera, v4l2 and Yavta do not have access to the Qualcomm camera firmware stack. For hardware accelerated encoding pipelines images captured with qtiqmmfsrc reach the filesink element in average in 6 ms. In the tests with 1280x720, 1920x1080 and 3840x2160, the board is able to maintain 30 frames per second.

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help .

Video Capture

Info

The following tests were done with an IMX577 camera connected to the Dragonwing IQ-9075 EVK

Enabling Camera Capture

Before enabling camera video capture, ensure that is is properly connected to the board. The setup for this connection may vary depending on the device. Once this is verified, enable the downstream camera driver by following the next steps:

1. Configure the camera overlay by running the following commands on the board:

echo -n "camx" > /var/data

efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -w -f /var/data

efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -p

Refer to the Dragonwing IQ-9075 EVK Camera Guide to verify the overlay hashes.

2. Sync the changes:

sync

3. Reboot the device:

reboot

4. Verify that the camera services are active by running the following command:

ps -ef | grep cam-server

5. Verify if the qtiqmmfsrc plugin is enabled:

gst-inspect-1.0 qtiqmmfsrc

If the qtiqmmfsrc plugin is disabled or missing, clear the GStreamer cache and try again:

rm ~/.cache/gstreamer-1.0/registry.aarch64.bin
gst-inspect-1.0 qtiqmmfsrc

Test Camera Capture

Camera output was captured using the following pipeline:

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! \
video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,\
interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc \
capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,\
video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-test.mp4

Resulting Output:

Output sample using `gst-play-1.0` to visualize file.

Performance Test

It is also possible to measure the board's performance while the camera captures. You can test the CPU and GPU load respectively with the following commands:

watch ps -o pid,cmd,%mem,rss -C gst-launch-1.0
watch -n 1 cat /sys/class/kgsl/kgsl-3d0/gpubusy

The pipeline for the performance measurements is:

gst-launch-1.0 -e -v qtiqmmfsrc camera=0 ! "video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1" ! v4l2h264enc ! h264parse ! mp4mux ! perf print-cpu-load=true ! filesink location=camera-test-perf.mp4

Also, measuring each element's latency, the results are as follows:

ELEMENT	COUNT	Average (ms)	Minimum (ms)	Maximum (ms)	P50 (ms)	P90 (ms)	P95 (ms)	P99 (ms)
v4l2h264enc0	289	5.523	3.753	15.784	5.506	5.912	6.006	6.261
mp4mux0	289	0.266	0.058	0.753	0.164	0.606	0.634	0.683
h264parse0	289	0.258	0.151	0.766	0.261	0.304	0.335	0.583
capsfilter0	289	0.070	0.044	0.162	0.069	0.073	0.097	0.117
Total Avg (ms)		6.116
Total Avg (ns)		6116486

Resolution	%MEM	RSS (KB)	BPS	Mean BPS	FPS	Mean FPS	CPU (%)
1280x720	0.2	78108	7779056.000	7820726.000	30.074	30.029	13
1920x1080	0.2	83648	15117488.000	15102548.000	29.956	30.074	13
3840x2160	0.2	90544	50627728.000	50033739.429	30.082	30.007	13

Other Camera Capture Frameworks

Attention

Camera capture was tested with other frameworks such as V4L2, libcam and YAVTA without any success. This may be attributed to how the board's device tree handles camera drivers and its different abstraction layers.

The tests for each framework are as follows:

libcam library

The first step is installing the libcam library in the ubuntu image (for Yocto builds, typically this package is already included):

sudo apt install libcamera-tools gstreamer1.0-libcamera

The libcamera elements should now be listed within the available Gstreamer elements:

gst-inspect-1.0 | grep libcamera:
libcamera:  libcameraprovider (GstDeviceProviderFactory)
libcamera:  libcamerasrc: libcamera Source

Once that is done, the available cameras were listed with the libcamera tools:

cam -l

[8:41:41.473836829] [64775]  WARN IPAManager ipa_manager.cpp:154 No IPA found in '/usr/lib/aarch64-linux-gnu/libcamera'
[8:41:41.473906565] [64775]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0
Available cameras:

Also, capture with the libcam Gstreamer element was tested:

gst-launch-1.0 libcamerasrc ! \
video/x-raw,width=1920,height=1080,framerate=30/1 ! \
v4l2h264enc ! h264parse ! mp4mux ! \
filesink location=libcamerasrc-test.mp4

Setting pipeline to PAUSED ...
[8:40:43.979474274] [64511]  WARN IPAManager ipa_manager.cpp:154 No IPA found in '/usr/lib/aarch64-linux-gnu/libcamera'
[8:40:43.979548384] [64511]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0
ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Could not find any supported camera on this system.
Additional debug info:
../src/gstreamer/gstlibcamerasrc.cpp(389): gst_libcamera_src_open (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:
libcamera::CameraMananger::cameras() is empty
ERROR: pipeline doesn't want to preroll.
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...

As you can see, the libcamera library cannot find any available/compatible devices, and therefore video camera capture is not supported for this platform.

V4L2

The V4L2 plugin comes with the Gstreamer installation mentioned previously. Also, to install the V4L2 tools set, run the following command:

sudo apt install v4l-utils

Trying to list the available devices with the V4L2 tools set results in the following output:

v4l2-ctl --list-devices

cam-req-mgr-devnode (no bus info):
	/dev/media1
	/dev/media2

msm_vidc_media (platform:aa00000.video-codec):
	/dev/media0

msm_vidc_decoder (platform:msm_vidc_bus):
	/dev/video32
	/dev/video33

Failed to open /dev/video0: Operation already in progress

Running a pipeline with the Gstreamer v4l2src element also fails as shown below:

gst-launch-1.0 -e v4l2src device=/dev/video0 ! "video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1" ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=v4l2src-test.mp4

Setting pipeline to PAUSED ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not open device '/dev/video0' for reading and writing.
Additional debug info:
../sys/v4l2/v4l2_calls.c(626): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Operation already in progress
ERROR: pipeline doesn't want to preroll.
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...

YAVTA

To install the YAVTA packages run:

sudo apt install yavta

Trying to run the YAVTA capturing command produces the following output:

 yavta -B capture-mplane -c -I -n 5 -f SRGGB10P -s 4056x3040 -F /dev/video0 --capture=5 --file='frame-#.raw'

Error opening device /dev/video0: Operation already in progress (114).

FAQ

Which camera was used in the current IQ-9075 capture test?: The IMX577 sensor.
Can I use Yavta, libcam or v4l2 to capture from the IMX577?: No, to capture from the IMX577, qtiqmmfsrc or another application compatible with Qualcomm's camera firmware stack must be used.
Is the capture and recording pipeline able to maintain 30 FPS with different resolutions?: Yes, for resolutions between 1280x720 and 3840x2160, provided that the hardware codec is used.

Related pages

❯

Share This Page

Send Email Using