View source for Video crop and scaling with DM816x and DM814x

<seo title="Video Scaling & Cropping with DM816x and DM814x" titlemode="replace" keywords="GStreamer, Linux SDK, Linux BSP,  Embedded Linux, Device Drivers, Nvidia, Xilinx, TI, NXP, Freescale, Embedded Linux driver development, Linux Software development, Embedded Linux SDK, Embedded Linux Application development, GStreamer Multimedia Framework."  description="This document presents a brief overview of the crop and scaling features available for the DM816x and DM814x platforms. Read this useful wiki page now."></seo>

= Introduction =

This document presents a brief overview of the crop and scaling features available for the DM816x and DM814x platforms.

Crop is oriented to the capability of ''cut'' a region of an image and just process this section in the next processing units whereas scaling is just referred to resize an image's size to fit a target width and height values. Those features can be done in the at either the capture stage or the display stage.

= Video capture crop and scaling =

The capture's crop and scaling features are part of the '''v4l2src'' GStreamer plugin and are exposed trough the ''crop-area'' property. This property is set following the next format:

 <pre style="background:#FFFFC0">crop-area=<left>,<top>@<width>x<height>&nbsp; </pre>

where ''left'' and ''top'' are the coordinates for the upper left corner of the crop region and the ''width'' and ''height'' values set its size. The figure 1 shows this concept. 

<blockquote>
'''NOTE:''' The ''left'', ''top'', ''width'' and ''height'' values '''MUST''' be set avoiding to request a region out of the input image's boundaries.
<\blockquote>

[[File:Crop_feature_01.png|400px|thumb|center|Figure 1. Crop-area basic concept]]

The ''crop-area'' feature was added together with a resizing feature where together execute the following actions:

* The crop feature ''cuts'' the input buffer and just considers the data from an specific region as shown in the figure 2. 

* If the capabilities after the v4l2src element are of the same size of the input buffer (the case shown) or just bigger than the cropped area, the output buffer will look such as "Output buffer (a)", the remaining space will be basically old data in memory. The following pipeline is an example of this setup.

 <pre style="background:#FFFFC0">VIDEO_STD=720P_60
CLEFT=20
CTOP=30
CWIDTH=640
CHEIGHT=480

gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 crop-area=$CLEFT,$CTOP@$CWIDTHx$CHEIHT ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=1280, height=720, framerate=60/1' ! \
omxbufferalloc numBuffers=12 ! \
omx_scaler ! \
v4l2sink sync=false
</pre>

* The resize feature is the one that actually determines the size of the output buffer, not the crop feature. It is worth to mention that the resizer is only able to do a downscaling process, upscaling is not supported by the architecture.

* If the width and height of the capabilities is the same that the ones specified in the crop-area parameter the output should look like "Output buffer (b)" where we have cropped the input video and the scaling feature has resized the output buffer to fit the cropped region. The following pipeline is an example of this setup.


 <pre style="background:#FFFFC0">VIDEO_STD=720P_60
CLEFT=100
CTOP=100
CWIDTH=640
CHEIGHT=480

gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 crop-area=$CLEFT,$CTOP@$CWIDTHx$CHEIHT ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=640, height=480, framerate=60/1' ! \
omxbufferalloc numBuffers=12 ! \
omx_scaler ! \
v4l2sink sync=false
</pre>

* If the ''crop-area'' parameter is omitted, v4l2src will use the width and height values from the pipeline capabilities and the left and top values will be set to 0. The following pipeline is an example of this setup.

 <pre style="background:#FFFFC0">
VIDEO_STD=720P_60

gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=640, height=480, framerate=60/1' ! \
omxbufferalloc numBuffers=12 ! \
omx_scaler ! \
v4l2sink sync=false
</pre>

[[File:Crop_feature_02.png|400px|thumb|center|Figure 2. Crop-area working model]]

= Video display crop =

If it is necessary to work with the input buffer on its original size but it is desired to crop it for display it is possible to play with the ''v4l2sink'' element's properties to do the trick as shown in the following pipeline:

 <pre style="background:#FFFFC0">
CAPS="video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1"
CLEFT=100
CTOP=100
CWIDTH=640
CHEIGHT=480

gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! capsfilter caps="$CAPS" ! \
omxbufferalloc numBuffers=12 ! gstperf ! omx_scaler  ! \
v4l2sink device=/dev/video1 crop-left=$CLEFT crop-top=$CTOP crop-width-$CWIDTH crop-height=$CHEIGHT
</pre>

where the ''left'', ''top'', ''width'' and ''height''properties work in the same way that in the v4l2src element.

= Video crop + upscaling =

Other feature available is crop plus upscaling, an use case for example would be to separate a 1080p frame in 4 subframes (960x540), and upscale each one of them into a larger frame (1080p).

[[File:Crop_upscale_feature_00.png|400px|thumb|center|Figure 3. Crop feature + upscale]]

Subframe 1

 <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,0@960x540 ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' \
! omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \
height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v
</pre>

Subframe 2

 <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,0@960x540 ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' \
! omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \
height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v
</pre>

Subframe 3

 <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,540@960x540 ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' ! \
omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \
height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v
</pre>
Subframe 4

 <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,540@960x540 ! \
'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' ! \
omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \
height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v
</pre>

= Video crop in the scaling stage =

It's also possible to do cropping and scaling at the same time in the middle of the pipeline, using the De-Interlacer or the Scaler OpenMax components.  Both upscaling and downscaling are supported, so it depends only of the user requirements about the display resolution or other specifications. For example, in the next pipeline the system captures 1080p video, then it crops the input to get the subframe 1, according with Figure 3; and finally it scales the video to be displayed at 720p.

<pre style="background:#FFFFC0">
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8  ! 'video/x-raw-yuv, \
format=(fourcc)NV12, width=1920, height=1080, framerate=(fraction)60/1' ! omxbufferalloc \
numBuffers=8 ! omx_mdeiscaler crop-area=0,0@960x540 name=d d.src_00 ! 'video/x-raw-yuv, \
width=(int)1280, height=(int)720' ! gstperf print-fps=true print-arm-load=true  !  v4l2sink \
device=/dev/video1 d.src_01 ! fakesink silent=true -v
</pre>

In the next pipeline, omx_scaler plugin is used instead of the omx_deiscaler plugin. In this case, the subframe 3 is gotten and it's scaled to 1080p again to be displayed.

<pre style="background:#FFFFC0">
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv, \
format=(fourcc)NV12, width=1920, height=1080, framerate=(fraction)60/1' !  omxbufferalloc \
numBuffers=8 ! omx_scaler crop-area=960,540@960x540 ! 'video/x-raw-yuv, width=(int)1920, \
height=(int)1080' ! gstperf print-fps=true print-arm-load=true  !  v4l2sink device=/dev/video1
</pre>

[[Category:DM8168]] [[Category:HowTo]] [[Category:Whitepaper]][[Category:DM8148]][[Category:RidgeRunTechnology]]