RidgeRun Auto exposure/Auto white balance library for DM368 and DM365

From RidgeRun Developer Connection

(Difference between revisions)
Jump to:navigation, search
m
m (Introduction)
Line 5: Line 5:
*Auto white balance automatically compensates color differences based on lighting so white actually appears white.
*Auto white balance automatically compensates color differences based on lighting so white actually appears white.
-
Some camera sensors don't include auto white balance and/or auto exposure processing, so RidgeRun offers a library with AE and AWB algorithms called '''librraew'''. This library was initially developed for the DM365/DM368 platform. The DM36x video processing front end (VPFE) has an H3A module designed to support control loops for auto focus, auto white balance and auto exposure by collecting statistics about the imaging/video data. There are two blocks in this module:
+
Some camera sensors don't include auto white balance and/or auto exposure processing, so RidgeRun offers a library with AE and AWB algorithms called '''librraew'''. This library was initially developed for the DM365/DM368 (DM36x) platform. The DM36x video processing front end (VPFE) has an H3A engine designed to support control loops for auto focus, auto white balance and auto exposure by collecting statistics about the imaging/video data. There are two blocks in this module:
* Auto focus engine  
* Auto focus engine  
* Auto exposure and auto white balance engine  
* Auto exposure and auto white balance engine  
-
The librraew library only uses the auto exposure and auto white balance hardware engine. This engine divides the frames into two dimensional blocks of pixels referred as windows. The engine can provide some image/video metrics:  
+
The librraew library only uses the auto exposure and auto white balance hardware engine. The DM36x does not allow the H3A engine to be used when the color space is YCbCr, which is common if you are using NTSC/PAL composite video input.
 +
 
 +
The H3A engine divides the frames into two dimensional blocks of pixels referred as windows. The engine provides image/video metrics:  
* Accumulation of clipped pixels along with all non-saturated pixels in each window on a per color basis.
* Accumulation of clipped pixels along with all non-saturated pixels in each window on a per color basis.
Line 16: Line 18:
* Minimum and maximum pixels values in each window on a per color basis.
* Minimum and maximum pixels values in each window on a per color basis.
-
The DM36x AE/AWB engine can be configured to use up to 36 horizontal windows with sum + {sum of squares or min+max} output or up to 56 horizontal windows with sum output. The AE/AWB engine can also be configure to use up to 128 vertical windows. The width and height for the windows is programmable.
+
The DM36x H3A engine can be configured to use up to 36 horizontal windows with sum + {sum of squares or min+max} output or up to 56 horizontal windows with sum output. The H3A engine can also be configure to use up to 128 vertical windows. The width and height for the windows is programmable.
The librraew library was tested using an Aptina MT9P031 CMOS sensor.  Support for other sensors was added later, thus validating the librraew design.  If you provide the appropriate sensor-specific functions for the library, it can work with any sensor. The implementation is a plain C library and can be re-used with and integrated with any application capable of making C function calls. [https://github.com/RidgeRun/ipiped/wiki Image Pipe Daemon] uses librraew to provide auto exposure/Auto white balance.
The librraew library was tested using an Aptina MT9P031 CMOS sensor.  Support for other sensors was added later, thus validating the librraew design.  If you provide the appropriate sensor-specific functions for the library, it can work with any sensor. The implementation is a plain C library and can be re-used with and integrated with any application capable of making C function calls. [https://github.com/RidgeRun/ipiped/wiki Image Pipe Daemon] uses librraew to provide auto exposure/Auto white balance.

Revision as of 19:29, 7 November 2011

Contents

Introduction

CMOS or CCD sensor video capture quality can be enhanced with image processing, like auto white balance (AWB) and auto exposure (AE) algorithms:

Some camera sensors don't include auto white balance and/or auto exposure processing, so RidgeRun offers a library with AE and AWB algorithms called librraew. This library was initially developed for the DM365/DM368 (DM36x) platform. The DM36x video processing front end (VPFE) has an H3A engine designed to support control loops for auto focus, auto white balance and auto exposure by collecting statistics about the imaging/video data. There are two blocks in this module:

The librraew library only uses the auto exposure and auto white balance hardware engine. The DM36x does not allow the H3A engine to be used when the color space is YCbCr, which is common if you are using NTSC/PAL composite video input.

The H3A engine divides the frames into two dimensional blocks of pixels referred as windows. The engine provides image/video metrics:

The DM36x H3A engine can be configured to use up to 36 horizontal windows with sum + {sum of squares or min+max} output or up to 56 horizontal windows with sum output. The H3A engine can also be configure to use up to 128 vertical windows. The width and height for the windows is programmable.

The librraew library was tested using an Aptina MT9P031 CMOS sensor. Support for other sensors was added later, thus validating the librraew design. If you provide the appropriate sensor-specific functions for the library, it can work with any sensor. The implementation is a plain C library and can be re-used with and integrated with any application capable of making C function calls. Image Pipe Daemon uses librraew to provide auto exposure/Auto white balance.

License

RidgeRun auto-exposure/auto-white-balance library (C) Copyright 2010 - RidgeRun LLC.

Evaluation and Development License

Subject to the terms and conditions of RidgeRun's SDK license, RidgeRun hereby grants to customer a product - based, non - exclusive, non - transferable, non - sublicensable, limited, worldwide license to install and use, for internal purposes only, an unlimited number of copies of the source and object code versions.

Distribution License

Subject to the terms and conditions of RidgeRun's SDK license, RidgeRun hereby grants to customer a non-exclusive, non-transferable, non-sublicensable, limited, worldwide license to distribute RidgeRun Software in object code format only (no source code) in one product model sold by the customer.

Algorithms

Auto white balance

When an image of a scene is captured by a digital camera sensor, the sensor response at each pixel depends on the scene illumination. Depending of the illumination, a distinct color cast appears over the captured scene. This effect appears in the captured image due to the color temperature of the light. If a white object is illuminated with a low color temperature light source, the object in the captured image will have a reddish tint. Similarly, when the white object is illuminated with a high color temperature light source, the object in the captured image will appear somewhat blue instead of pure white. The human eye compensates for color cast automatically through a characteristic known as color constancy, allowing the colors to be independent of the illumination. Auto white balance tries to simulate the color constancy for captured images.

Many auto white balance algorithms follow a two-stage process:

Auto exposure

One of the main problems affecting image quality, leading to disappointing pictures, comes from improper light exposure. The image exposure is the amount of light that reaches the sensor. Exposure determines the lightness or darkness of the resulting image. If too much light strikes the image sensor, the image will be overexposed, washed out, and faded. If too little light reaches the camera sensor produces an underexposed image, dark and lacking in details especially in shadow areas. Auto exposure (AE) algorithms adjust the captured image in an attempt to reproduce the most important regions (according to contextual or perceptive criteria) with an average level of brightness, more or less in the middle of the possible range.

Auto exposure algorithms involves three processes:

Documentation

Using the demo version of librraew

You can request a demo version of librraew in order to test the auto-white balance and auto-exposure algorithms to see if the technology meets your needs. This library will allow you to use all the features that comes with the full version of the librraew but with the following limitations:

References

  1. Battiato, G. Messina, and A. Castorina. Exposure correction for imaging devices: an overview. In Single-Sensor Imaging: Methods and Applications for Digital Cameras, chapter 12. Rastislav Lukac, October 2008.
  2. Lee J.S., Jung Y.Y, Kim B.S., and Ko S.J. An advanced video camera system with robust af, ae, and awb control. IEEE Transactions on Consumer Electronics, 47:694–699, August 2001
  3. Edmund Y. Lam. Combining gray world and retinex theory for automatic white balance in digital photography. Consumer Electronics, 2005. (ISCE 2005). Proceedings of the Ninth International Symposium on, pages 134–139, June 2005.
  4. Edmund Y. Lam and George S. K. Fung. Automatic white balancing in digital photography. In Single-Sensor Imaging: Methods and Applications for Digital Cameras. Taylor & Francis Group, LLC, 2009.
  5. Nitin Sampat, Shyam Venkataraman, Thomas Yeh, and Robert L. Kremens. System implications of implementing auto-exposure on consumer digital cameras. Proc. SPIE. Sensors, Cameras, and Applications for Digital Photography, 3650:100–107, March 1999
Navigation
Toolbox