Video crop and scaling with DM816x and DM814x

Introduction

This document presents a brief overview of the crop and scaling features available for the DM816x and DM814x platforms.

Crop is oriented to the capability of cut a region of an image and just process this section in the next processing units whereas scaling is just referred to resize an image's size to fit a target width and height values. Those features can be done in the at either the capture stage or the display stage.

Video capture crop and scaling

The capture's crop and scaling features are part of the 'v4l2src GStreamer plugin and are exposed trough the crop-area property. This property is set following the next format:

crop-area=<left>,<top>@<width>x<height>

where left and top are the coordinates for the upper left corner of the crop region and the width and height values set its size. The figure 1 shows this concept.

NOTE: The left, top, width and height values MUST be set avoiding to request a region out of the input image's boundaries. <\blockquote>

Figure 1. Crop-area basic concept

The crop-area feature was added together with a resizing feature where together execute the following actions:

• The crop feature cuts the input buffer and just considers the data from an specific region as shown in the figure 2.

• If the capabilities after the v4l2src element are of the same size of the input buffer (the case shown) or just bigger than the cropped area, the output buffer will look such as "Output buffer (a)", the remaining space will be basically old data in memory. The following pipeline is an example of this setup.

VIDEO_STD=720P_60
CLEFT=20
CTOP=30
CWIDTH=640
CHEIGHT=480

gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 crop-area=$CLEFT,$CTOP@$CWIDTHx$CHEIHT ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=1280, height=720, framerate=60/1' ! \
omxbufferalloc numBuffers=12 ! \
omx_scaler ! \
v4l2sink sync=false

• The resize feature is the one that actually determines the size of the output buffer, not the crop feature. It is worth to mention that the resizer is only able to do a downscaling process, upscaling is not supported by the architecture.

• If the width and height of the capabilities is the same that the ones specified in the crop-area parameter the output should look like "Output buffer (b)" where we have cropped the input video and the scaling feature has resized the output buffer to fit the cropped region. The following pipeline is an example of this setup.

VIDEO_STD=720P_60
CLEFT=100
CTOP=100
CWIDTH=640
CHEIGHT=480

gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 crop-area=$CLEFT,$CTOP@$CWIDTHx$CHEIHT ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=640, height=480, framerate=60/1' ! \
omxbufferalloc numBuffers=12 ! \
omx_scaler ! \
v4l2sink sync=false

• If the crop-area parameter is omitted, v4l2src will use the width and height values from the pipeline capabilities and the left and top values will be set to 0. The following pipeline is an example of this setup.

VIDEO_STD=720P_60

gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=640, height=480, framerate=60/1' ! \
omxbufferalloc numBuffers=12 ! \
omx_scaler ! \
v4l2sink sync=false

Figure 1. Crop-area working model

Video display crop

If it is necessary to work with the input buffer on its original size but it is desired to crop it for display it is possible to play with the v4l2sink element's properties to do the trick as shown in the following pipeline:

CAPS="video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1"
CLEFT=100
CTOP=100
CWIDTH=640
CHEIGHT=480

gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! capsfilter caps="$CAPS" ! \
omxbufferalloc numBuffers=12 ! gstperf ! omx_scaler  ! \
v4l2sink device=/dev/video1 crop-left=$CLEFT crop-top=$CTOP crop-width-$CWIDTH crop-height=$CHEIGHT

where the left, top, width and heightproperties work in the same way that in the v4l2src element.

Video crop + upscaling

Other feature available is crop plus upscaling, an use case for example would be to separate a 1080p frame in 4 subframes (960x540), and upscale each one of them into a larger frame (1080p).

Figure 3. Crop feature + upscale

Subframe 1

gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,0@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=8 \  ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v

Subframe 2

gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,0@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=8 \  ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v

Subframe 3

gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,540@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=8 \  ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v

Subframe 4

gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,540@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! omxbufferalloc numBuffers=8 \ ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v