Capturing JPEG images at full (allowed) resolution on DM36x platform

From RidgeRun Developer Connection

Revision as of 20:53, 21 November 2011 by Mmadrigal (Talk | contribs)
Jump to:navigation, search

Contents

Introduction

This whitepaper shows how to configure the SDK to capture the maximum allowed image size on the LeopardBoard DM36x. When this document was wrote the maximum allowed resolution for still images was of 2176x1944 pixels.

Hardware used

Configuring the SDK to capture 2176x1944 still images

Our first goal is to get a simple raw data capture from the sensor with a fixed resolution of 2176x1944 pixels, so we can start with the following pipeline:

gst-launch --gst-debug=v4l2:5 v4l2src always-copy=false chain-ipipe=true num-buffers=1 ! \
'video/x-raw-yuv,format=(fourcc)UYVY, width=2176, height=1944' ! fakesink

On first run we should get a segmentation fault error like this:

/ # gst-launch -e v4l2src always-copy=false num-buffers=1 chain-ipipe=true ! 'vi
deo/x-raw-yuv,format=(fourcc)UYVY, width=2176, height=1944' ! dmaiaccel ! dmaien
c_jpeg ! filesink location=test_dmai_3.jpeg
Setting pipeline to PAUSED ...
davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:124
vpfe-capture vpfe-capture: IPIPE Chained
vpfe-capture vpfe-capture: Resizer present
vpfe-capture vpfe-capture: standard not supported
vpfe-capture vpfe-capture: dma_alloc_coherent size 8462336 failed
Segmentation fault

The line vpfe-capture vpfe-capture: dma_alloc_coherent size 8462336 failed indicates there is not enough space in the capture/input buffer.

Now if we run the same pipeline with a little bit more of debug we can see something like that:

gst_v4l2_object_probe_caps_for_format:<v4l2src0> done iterating discrete frame sizes
gst_v4l2_object_set_format:<v4l2src0> Setting format to 2176x1944, format UYVY
gst_v4l2_buffer_pool_new:<v4l2src0> STREAMING, requesting 2 MMAP buffers
gst_v4l2_buffer_pool_new:<v4l2src0>  count:  3
gst_v4l2_buffer_pool_new:<v4l2src0>  type:   1
gst_v4l2_buffer_pool_new:<v4l2src0>  memory: 1
gst_v4l2_buffer_pool_new:<v4l2src0> using 3 buffers instead
gst_v4l2_buffer_new:<v4l2src0> creating buffer 0, 0x2aac0 in pool 0x1ee70
gst_v4l2_buffer_new:<v4l2src0>   index:     0
gst_v4l2_buffer_new:<v4l2src0>   type:      1
gst_v4l2_buffer_new:<v4l2src0>   bytesused: 0
gst_v4l2_buffer_new:<v4l2src0>   flags:     00000000
gst_v4l2_buffer_new:<v4l2src0>   field:     0
gst_v4l2_buffer_new:<v4l2src0>   memory:    1
gst_v4l2_buffer_new:<v4l2src0>   MMAP offset:  0
gst_v4l2_buffer_new:<v4l2src0>   length:    8460288
gst_v4l2_buffer_new:<v4l2src0>   input:     0
gst_v4l2_buffer_new:<v4l2src0> creating buffer 1, 0x2aa20 in pool 0x1ee70
gst_v4l2_buffer_new:<v4l2src0>   index:     1
gst_v4l2_buffer_new:<v4l2src0>   type:      1
gst_v4l2_buffer_new:<v4l2src0>   bytesused: 0
gst_v4l2_buffer_new:<v4l2src0>   flags:     00000000
gst_v4l2_buffer_new:<v4l2src0>   field:     0
gst_v4l2_buffer_new:<v4l2src0>   memory:    1
gst_v4l2_buffer_new:<v4l2src0>   MMAP offset:  8462336
gst_v4l2_buffer_new:<v4l2src0>   length:    8460288
gst_v4l2_buffer_new:<v4l2src0>   input:     0
gst_v4l2_buffer_new: Failed to mmap: Cannot allocate memory
gst_v4l2_buffer_finalize:<v4l2src0> finalizing buffer 0x2aa20 1
gst_v4l2_buffer_finalize:<v4l2src0> the pool is shutting down
gst_v4l2_buffer_finalize:<v4l2src0> buffer 0x2aa20 (data (nil), len 8460288) not recovered, unmapping

As we can see it is not possible to allocated the 3 required buffers, where each one have a size equal to:

BUFF_SIZE = W*H*2

In theory we just need to increase the capture buffer size to 3 times this value, but in practice it was found that we require the double of memory space:

CAPTURE_BUFF_SIZE = W*H*2*6 = 2176*1944*2*6 = 50761728

We can set this value on

 -> Architecture configurations
    -> Maximum Video Input Buffer Size

setting this option to Other (enter buffer size) and setting the Buffer size for v4l2 input field to this value.

Since we increased the amount of memory for capture we need to modify the memory map to match this changes. The DM36x memory map is ordered as is shown below:

--------------------------------------
                          CMEM
    RESERVED   -----------------------
                 CAPTURE_DISPLAY_MEM
--------------------------------------
    KERNEL
--------------------------------------

This way we have two main memory block named KERNEL and RESERVED. The reserved memory allocates both CMEM and capture/display buffers. So if we increase one memory region all others must change to keep the same memory size.

If we use the default SDK configuration we are using Composite output, this mean a display buffers size of 3145728 as we can see from the bootargs on the variable:

davinci_display.cont2_bufsize=3145728

This way we need around 52M of memory for the CAPTURE_DISPLAY_MEM space since:

CAPTURE_BUF_SIZE + DISLPLAY_BUF_SIZE 
50761728 + 3145728
53907456 ~ 52M

Assuming we are using the default CMEM amount (15M) we can calculate the amount of reserved memory we need as

RESERVED = CAPTURE_DISPLAY_MEM + CMEM
RESERVED = 52M + 15M = 67M

So we should the the option

-> Proprietary software
   -> Amount of reserved memory from the kernel

to 67M = 0x4300000.

This way the SDK will automatically resize the kernel memory to match with the specified reserved memory.

Now, once we rebuild out SDK we got the following output when running the pipeline:

/ # gst-launch v4l2src always-copy=false chain-ipipe=true num-buffers=1 ! 'video
/x-raw-yuv,format=(fourcc)UYVY, width=2176, height=1944' ! fakesink 
Setting pipeline to PAUSED ...
davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:124
vpfe-capture vpfe-capture: IPIPE Chained
vpfe-capture vpfe-capture: Resizer present
vpfe-capture vpfe-capture: standard not supported
Pipeline is live and does not need PREROLL ...
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to set norm for device '/dev/video0'.
Additional debug info:
../../../src/sys/v4l2/v4l2_calls.c(743): gst_v4l2_set_norm (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Invalid argument
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Video input device did not accept new frame rate setting.
Additional debug info:
../../../src/sys/v4l2/v4l2src_calls.c(342): gst_v4l2src_set_capture (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Invalid argument
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 84000331 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
/ # 

Which represents a correct output. If we run this pipeline with debug options we should be able to see a section like this:

gst_v4l2_object_probe_caps_for_format:<v4l2src0> done iterating discrete frame sizes
gst_v4l2_object_set_format:<v4l2src0> Setting format to 2176x1944, format UYVY
gst_v4l2_buffer_pool_new:<v4l2src0> STREAMING, requesting 2 MMAP buffers
gst_v4l2_buffer_pool_new:<v4l2src0>  count:  3
gst_v4l2_buffer_pool_new:<v4l2src0>  type:   1
gst_v4l2_buffer_pool_new:<v4l2src0>  memory: 1
gst_v4l2_buffer_pool_new:<v4l2src0> using 3 buffers instead
gst_v4l2_buffer_new:<v4l2src0> creating buffer 0, 0x2aac0 in pool 0x1ee70
gst_v4l2_buffer_new:<v4l2src0>   index:     0
gst_v4l2_buffer_new:<v4l2src0>   type:      1
gst_v4l2_buffer_new:<v4l2src0>   bytesused: 0
gst_v4l2_buffer_new:<v4l2src0>   flags:     00000000
gst_v4l2_buffer_new:<v4l2src0>   field:     0
gst_v4l2_buffer_new:<v4l2src0>   memory:    1
gst_v4l2_buffer_new:<v4l2src0>   MMAP offset:  0
gst_v4l2_buffer_new:<v4l2src0>   length:    8460288
gst_v4l2_buffer_new:<v4l2src0>   input:     0
gst_v4l2_buffer_new:<v4l2src0> creating buffer 1, 0x2aa20 in pool 0x1ee70
gst_v4l2_buffer_new:<v4l2src0>   index:     1
gst_v4l2_buffer_new:<v4l2src0>   type:      1
gst_v4l2_buffer_new:<v4l2src0>   bytesused: 0
gst_v4l2_buffer_new:<v4l2src0>   flags:     00000000
gst_v4l2_buffer_new:<v4l2src0>   field:     0
gst_v4l2_buffer_new:<v4l2src0>   memory:    1
gst_v4l2_buffer_new:<v4l2src0>   MMAP offset:  8462336
gst_v4l2_buffer_new:<v4l2src0>   length:    8460288
gst_v4l2_buffer_new:<v4l2src0>   input:     0
gst_v4l2_buffer_new:<v4l2src0> creating buffer 2, 0x2a980 in pool 0x1ee70
gst_v4l2_buffer_new:<v4l2src0>   index:     2
gst_v4l2_buffer_new:<v4l2src0>   type:      1
gst_v4l2_buffer_new:<v4l2src0>   bytesused: 0
gst_v4l2_buffer_new:<v4l2src0>   flags:     00000000
gst_v4l2_buffer_new:<v4l2src0>   field:     0
gst_v4l2_buffer_new:<v4l2src0>   memory:    1
gst_v4l2_buffer_new:<v4l2src0>   MMAP offset:  16924672
gst_v4l2_buffer_new:<v4l2src0>   length:    8460288
gst_v4l2_buffer_new:<v4l2src0>   input:     0
gst_v4l2_buffer_pool_qbuf:<v4l2src0> enqueue pool buffer 0
gst_v4l2_buffer_pool_qbuf:<v4l2src0> num_live_buffers--: 2
gst_v4l2_buffer_pool_qbuf:<v4l2src0> enqueue pool buffer 1
gst_v4l2_buffer_pool_qbuf:<v4l2src0> num_live_buffers--: 1
gst_v4l2_buffer_pool_qbuf:<v4l2src0> enqueue pool buffer 2
gst_v4l2_buffer_pool_qbuf:<v4l2src0> num_live_buffers--: 0
Pipeline is live and does not need PREROLL ...

Here we can see how this time it was possible to allocated the 3 buffers.

Adding encoding stage

Next step on this paper is to add the JPEG encoding, for this we will use the following pipeline:

FILE_NAME=test.jpeg

gst-launch -e v4l2src always-copy=false num-buffers=1 chain-ipipe=true ! 'video/x-raw-yuv,format=(fourcc)UYVY, \
width=2176, height=1944' ! dmaiaccel ! dmaienc_jpeg ! filesink location=$FILE_NAME

This time we should get an error like this:

/ # gst-launch -e v4l2src always-copy=false num-buffers=1 chain-ipipe=true ! 'vi
deo/x-raw-yuv,format=(fourcc)UYVY, width=2176, height=1944' ! dmaienc_jpeg ! fil
esink location=$FILE_NAME
Setting pipeline to PAUSED ...
davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:124
vpfe-capture vpfe-capture: IPIPE Chained
vpfe-capture vpfe-capture: Resizer present
vpfe-capture vpfe-capture: standard not supported
Pipeline is live and does not need PREROLL ...
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to set norm for device '/dev/video0'.
Additional debug info:
../../../src/sys/v4l2/v4l2_calls.c(743): gst_v4l2_set_norm (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Invalid argument
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Video input device did not accept new frame rate setting.
Additional debug info:
../../../src/sys/v4l2/v4l2src_calls.c(342): gst_v4l2src_set_capture (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Invalid argument
Setting pipeline to PLAYING ...
New clock: GstSystemClock
CMEMK Error: Failed to find a pool which fits 25380864
CMEM Error: getPool: Failed to get a pool fitting a size 25380864
Failed to allocate memory.
ERROR: from element /GstPipeline:pipeline0/dmaienc_jpeg:dmaienc_jpeg0: No space left on the resource.
Additional debug info:
../../src/src/gsttidmaienc.c(1227): get_raw_buffer (): /GstPipeline:pipeline0/dmaienc_jpeg:dmaienc_jpeg0:
failed to create input buffers
Execution ended after 97345749 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...

(gst-launch-0.10:1111): GStreamer-CRITICAL **: gst_mini_object_unref: assertion `GST_IS_MINI_OBJECT (mini_object)' failed
Setting pipeline to NULL ...
Freeing pipeline ...

As we can see on the following lines

CMEMK Error: Failed to find a pool which fits 25380864
CMEM Error: getPool: Failed to get a pool fitting a size 25380864

There is not enough CMEM space for the encoding process, so we need to increase CMEM enough to be more or equal to 25380864 bytes, that is, we need around of 25M = 0x1834800. In order to give CMEM a little bit of memory space we set its size to

CMEM = 0x1900000 

on

-> Proprietary software
   -> Amount of reserved memory for cmemk

Since we have modified the CMEM amount we need to increase the RESERVED memory the same amount we increased CMEM, this way we got that the required RESERVED memory must be

RESERVED = 0x4D00000

which must be set on

-> Proprietary software
   -> Amount of reserved memory from the kernel

After that we can rebuilt our SDK and run the pipeline without problem.

Navigation
Toolbox