Birds Eye View - Overview
| ⇦ Getting Started/How to Get the Code | Home | How BEV Works/Background ⇨ |
 |
| Getting Started |
|---|
| How BEV Works |
| User Guide |
| Calibration Guide |
| Calibration Guide Legacy |
| GStreamer |
| Performance |
| Contact Us |
RidgeRun's Birds Eye View enables you to generate aerial views. You can transform multiple camera views surrounding an object into a single top-down elevated ground-plane view, similar to what a bird flying on top of the object would see when looking down. This page provides an overview of how RidgeRun's Birds Eye View generates the elevated views from camera inputs.
Consider Fig. 1, four cameras are attached to a vehicle's capturing the front, rear, left and right perspectives surrounding it. Birds Eye View takes these images as input and generates a single elevated view above the vehicle surroundings.
The Birds Eye View is a virtual composition formed by a collection of images that capture the surrounding perspectives of an object. Notice that the cameras not necessarily are facing down. The aerial view is generated from the limited floor information provided by the camera views, after performing a carefully crafted perspective transformation.
To create good elevated views, the camera views must have some overlap between them. This allows the system to generate a full Birds Eye View image, without gaps of missing information. The center of a generated Birds Eye View image, is typically an empty region, since there are no cameras facing the object. Commercial applications typically overlay an avatar of the object on top of it. In the example presented in Fig. 1, the vehicle is represented with a black car avatar.
Birds Eye View images have become very popular, especially among terrestrial vehicles. It is common to see them on modern cars, where they are used by the driver in order to have better control of the car's dimensions and the obstacles in the exterior. Autonomous robots also use them to simplify collision avoidance and path planning algorithms. Lately, heavy machinery vehicles use them to have a complete view of the surroundings and avoid accidents, which otherwise would be impossible due to the size of the system.
Birds Eye View Workflow
A typical Birds Eye View workflow starts with 3-6 camera inputs and concludes with a single elevated top-down transformed perspective image. Fig 2. presents the general processing path used by a system running RidgeRun's Birds Eye View and Fig. 3 offers a visual aid to understand the perspective transformations and image resizing steps.
- Capture camera frames: Obtain frames from the video sources that surround the object of interest. For example: 4 cameras with views facing away from the object and each with 180 degrees field of view (FOV).
- Resize frames: Resize input frames to the desired resolution for further processing.
- Remove lens distortion: Apply lens undistortion algorithm to remove the fisheye effect, see [Undistortion] for more. The perspective transformation needs to be performed on rectilinear images. This means that, in order to use fisheye lens cameras, you need to remove this distortion first. Even if the image was not captured using a fisheye lens, perspective cameras still have some slightly noticeable curvature that may be corrected. Rectifying these images results in higher quality BEV images.
- Perspective transformation: Generate a top-down perspective of the image view.
- Perspective mapping: Map the points from the original image view to the top-down perspective, see Background for more.
- Enlarging: Enlarge the image to include the region of interest (ROI).
- Cropping: Crop the output of the enlargement process to the size required for the final image creation.
- Create a final top view image: Combine the four camera views into a single top-down perspective output image.
As presented in Fig. 3, the input image is transformed into a top-down view using perspective transformation and mapping, then enlarged to include only the region of interest, this enlargement helps correct the aspect ratio. Finally, the image is cropped, this serves two purposes: on the one hand, it removes sections where there is no information available, and on the other hand, it removes the parts of the image where stretching is too noticeable due to extreme pixel interpolation. Both of these effects are caused by the perspective transformation process.
Fig. 4 presents the perspective transformation process internal workflow with an image captured with a camera.
For the math behind Birds Eye View, see Background.
FAQ
- What is RidgeRun Birds Eye View?
- RidgeRun Birds Eye View is a system that transforms multiple camera views around an object into a single top-down elevated ground-plane view.
- How does Birds Eye View work?
- The workflow is: capture camera frames, resize the frames, remove lens distortion, perform perspective transformation, enlarge the image to include the region of interest, crop it to the required size, and combine the resulting views into one final top-down image.
- What does the Birds Eye View output look like?
- The output is a single elevated top-down view of the surroundings, similar to what a bird would see when looking down on the object.
- Do Birds Eye View cameras need to face downward?
- No. The cameras do not necessarily face down. The aerial view is generated from the available floor information after a perspective transformation.
- Why do the camera views need overlap?
- The cameras must overlap so the system can generate a full Birds Eye View image without gaps of missing information.
- Why is the center of the generated Birds Eye View often empty?
- The center is typically empty because there are no cameras facing the object itself.
- Why is lens undistortion in the workflow?
- Lens distortion is removed to correct the fisheye effect before the perspective transformation stage. This is only required for cameras with fisheye lens.
- What happens during perspective transformation?
- The perspective transformation generates a top-down perspective of the image view. This stage includes perspective mapping, enlarging, and cropping.