RidgeRun Auto exposure/Auto white balance library for DM368 and DM365

From RidgeRun Developer Connection

(Difference between revisions)
Jump to:navigation, search
Line 33: Line 33:
= Design =
= Design =
-
Three applications are required to support the auto exposure and auto white-balance (AEW) adjustments:
+
Three applications are required to support the auto exposure and auto white-balance (AEW) adjustments {TAG(tag=>sup)}super{TAG}:
*Ipiped, a D-Bus server for controlling and configuring the camera sensor, the dm365 video processor and the aew library.
*Ipiped, a D-Bus server for controlling and configuring the camera sensor, the dm365 video processor and the aew library.
*Librraew, a library that includes auto white balance and auto exposure algorithms.
*Librraew, a library that includes auto white balance and auto exposure algorithms.

Revision as of 22:49, 14 April 2011

Contents

Introduction

Video capture quality can be enhance with image processing, like auto white balance(AWB) and auto exposure algorithms(AE):

Some camera sensors don't include auto white balance and/or auto exposure processing, so RidgeRun offers a library with AE and AWB algorithms called librraew. This library has been developed for the DM365 platform. The DM365 VPFE has a module H3A designed to support control loops for auto focus, auto white balance and auto exposure by collecting statistics about the imaging/video data. There are two blocks in this module:

The librraew only use the auto exposure and auto white balance engine. This engine divides the frames into two dimensional blocks of pixels referred as windows. The engine can provide some image/video metrics:

The AE/AWB engine can be configured to use up to 36 horizontal windows with sum + {sum of squares or min+max} output or up to 56 horizontal windows with sum output. Also can be configure to use up to 128 vertical windows. The width and height for the windows is programmable.

By now, the librraew has only been tested for the camera sensor mt9p031 but if you provide the appropriate functions for the library, it can works with any sensor.

Auto white balance

When an image of a scene is captured by a digital camera sensor, the sensor response at each pixel depends on the illumination. Depending of the scene illumination, a distinct color cast appears over the captured scene. This effect appears in the captured image due to the color temperature of the light. If a white object is illuminated with a low color temperature light source, the object in the captured image will be reddish. Similarly, when the white object is illuminated with a hight color temperature light source, the object in the captured image will be bluish. The human eye is capable of compensate this color cast automatically through a characteristic known as color constancy, allowing the colors to be independent of the illumination. Auto white balance tries to simulate the color constancy for images capture.

Many AWB algorithms follow a two-stage process:

Bellow a brief description of the AWB algorithms for the librraew is presented:

Gray World
The Gray World algorithm is based on the assumption that given an image with sufficient amount of color variation, the average reflectance of the scene is achromatic(gray). To be achromatic, the mean of the red, green, and blue channels in a given scene should be roughly equal. If the scene has a color that dominates, the results of the algorithm may not be satisfactory.
White Patch
The White Patch algorithm assumes that the maximum response in an image is caused by a perfect reflector, that represents the color of the illumination. This white balancing attempts to equalize the maximum value of the three channels to produce a white patch. This algorithm must be used on scenes that ensure the captured image does not include saturated pixels.

Auto exposure

Design

Three applications are required to support the auto exposure and auto white-balance (AEW) adjustments {TAG(tag=>sup)}super{TAG}:

Running Ipiped

Ipiped must run in background:

 ipiped &

Ipiped registers with D-Bus and waits until ipipe-client requests to execute a method.

Running Ipipe-client

Ipipe-client is a Dbus client that use commands to invoke methods of the ipiped, so ipiped must be running to use ipipe-client. A command can required arguments depending of the functionality. Ipipe-client has two operation modes, you can ask to execute a single command or you can open an interactive console to execute a group of commands.

To execute a single command, you can use the following command line syntax

 ipipe-client <command> <argument 1> ... <argument n>

To get into the interactive console, you have to run ipipe-client without any command. Then to execute a command you only need to use the command and the required arguments.

ipipe-client
ipipe-client$ <command 1> <argument 1> ... <argument n>
ipipe-client$ <command 2> <argument 1> ... <argument n>

To quit the interactive console you can use quit or exit.

In order to know the commands that are available run:

ipipe-client help

or get into the interactive console and execute help.

This shows a description of each command, as follows:

Command                         Description

help                    Displays the help text for all the possible commands or a specific command.
set-debug               Enable/Disable debug messages.
init-aew                Initialize AEW algorithms.
stop-aew                End AEW algorithm.
shell                   Execute a shell command(shell_cmd) using interactive console.
ping                    Show if ipipe-daemon is alive.
quit                    Quit from the interactive console.
exit                    Exit from the interactive console.
get-video-processor     Show the video processor that is being used.
get-sensor              Show the sensor that is being used.
run-config-script       Execute a group of ipipe-client commands.
set-previewer-mode      Configure previewer on continuous or one-shot mode.
set-bayer-pattern       Sets R/Gr/Gb/B color pattern to the previewer.
set-digital-gain        Sets red (R), green (G) and blue gains (G) on the ipipe.
get-digital-gain        Returns the gain value for each color component(RGB).
set-luminance           Brightness(Br) and contrast(C) adjustment.
get-luminance           Returns the value of the Brightness(Br) and contrast(C) adjustment.
flip-vertical           Flips the image vertically(on the sensor).
flip-horizontal         Flips the image horizontally (on the sensor).
set-exposure            Sets the effective shutter time  of the sensor for the light integration.
get-exposure            Gets the exposure time of the sensor in us.
set-sensor-gain         Sets red(R), green(G) and blue(B) gain directly on the sensor.
get-sensor-gain         Gets sensor red(R), green(G) and blue(B).

If you want more detailed information about a command execute:

ipipe-client help <command> 

Running Librraew

Auto exposure and auto white balance adjustments can be started with an ipipe-client's command called init-aew. Init-aew requires some arguments to define the algorithms and other parameters. To see the arguments required you can request for help that show you the list as follows:

Command: init-aew
Syntax: init-aew <WB> <AE> <G> <EM> <T[us]> <fps> <seg> <width> <height>
Description: Initialize AEW algorithms                                  
Arguments:                                                              
        WB: white balance algorithm, the options are:                   
                G -for gray world algorithm                             
                W -for retinex algorithm                                
                N -for none                                             
        AE: auto exposure algorithm, the options are                    
                EC -for electronic centric                              
                N -for none                                             
        G: gain type, the options are:                                  
                S -to use the sensor gain                               
                D -to use the digital
        EM: exposure metering method, the options are:
                S -for spot metering that take into account the light
                information of a small portion in the center of the
                frame and the rest is ignored
                P -for partial metering that take into account the
                light information of a larger portion in the center
                of the frame (10% - 15%) and the rest of the frame is ignored.
                C -for center weighted metering that take into account
                the light information coming from the entire frame with
                emphasis placed on the center area
                A -for average metering that take into account the light
                information from the entire frame without weighting
                SG -for segmented metering that divides the frame
                on 6 pieces and weighting them to avoid backlighting
        T: wait time in us, specifies the time between
                algorithm adjustments, max value=1s=1000000us
        fps: minimum frame rate
        seg: frame segmentation factor, each frame is segmented into
                regions, this factor represents the percentage of the
                maximum number of possible regions
        width: captured video/image horizontal size
        height: captured video/image vertical size

Also you can stop automatic adjustments with the command stop-aew

Some of the init-aew arguments need to be explained in more detail:

Navigation
Toolbox