Birds Eye View - Bird's Eye View Startup Guide

From RidgeRun Developer Wiki



⇦ Getting_Started Home Getting_Started/Minimal_Application ⇨


Getting Started
User Guide
Calibration Guide
GStreamer
Performance Measurements
Contact Us





Prerequisites

  • Four fisheye cameras ready to use
  • Cameras mounted in a car, boat, device, table, etc, in final distribution that will be used
  • Completion of Installation Guide

Summary

  1. Cameras layout setup: create and put the cameras in correct position for desired application like a car, boat or any other device.
  2. Cameras distortion parameters calibration: get the K/D distortion parameters for each camera to correctly remove fisheye from image.
  3. Top view image calibration: using a chessboard calibration pattern the birds eye view calibration process is fast, with easy fine-tuning capabilities.
  4. Run application: if the cameras layout does not change, previous steps are only made once, after that, you will be able to run your applications with same configuration.

Auto Calibration Script

The RidgeRun's Bird's Eye View library provides a calibration script that will help in finding camera distortion parameters. Navigate to the following directory: 

cd birds-eye-view/prototype

To check the python script allowed parameters the following command can be used: 

python3 autoCalibration.py -h

The options are:

Table 1. Calibration script options

Parameter Short Parameter Description Arguments
--help -h Shows the help message and exit -
--load -l Load current settings file and show preview -
--debug -d Show debug FPS information for preview -
--remove -r Allow to remove calibration images from filesystem -
--skip -s Skip saving fisheye images and calibrate with current
images that are on the filesystem		 -
--imagesPath -p Path to the directory where the calibration images
will be saved		 -
--json -j Output json file to save the calibration values to Path with file name to save calibration
settings in JSON format.

A. Cameras Layout Setup

Put the cameras in the final setup where they will be used, for example, a Car, Boat, other devices. In this wiki an indoor setup with a small table as camera holder will be used. Label each camera and make sure to know which Linux device is bound for each camera.

Table 2. Cameras device mapping

Camera Device
Front /dev/video2
Left /dev/video4
Right /dev/video6
Back /dev/video8

Add the information to the autoCalibrationSettings file: 

vim birds-eye-view/prototype/autoCalibrationSettings.json

Add the information of the resolution size for the cameras, input framerate, and output size of frames 

...
    "camerasWidth": 1920,
    "camerasHeight": 1080,
    "framerate": 30,
    "outputWidth": 600,
    "outputHeight": 300,
...

There is a node inside the "views" node in the JSON file for each camera where you will need to add the device ID value. For example: 

...
"views": [
        {
            "tag": "FrontCamera",
            "deviceID": 2,
...


B. Cameras Distortion Parameters Calibration

In this step, the user has two options:

OPTION 1: Camera parameters are already available

The floating-point camera matrix, which is defined by:

And the vector of distortion coefficients defined by:

The information for each camera can be added to same node in the autoCalibrationSettings.json file, for example: 

...
"views": [
        {
            "tag": "FrontCamera",
            "deviceID": 2,
            "balance": 0,
            "camK": [[533,0.0,954],[1.0,530,541],[0.0,0.0,1.0]],
            "camD": [[-0.07808330315826352],[0.06710634719219645],[-0.048871815691283065],[0.011097077754507887]],
...

Continue with next step C. Top View Image Calibration

OPTION 2: Camera parameters need to be automatically obtained

If the camera parameters are not available, the auto-calibration script can get several calibration images and compute the resulting parameters. A chessboard pattern is needed, several are available through the internet for example here the user can find several chessboard patterns for different printing sizes. Once a chessboard pattern has been selected, add the inner corner width and height number to the autoCalibrationSettings.json 

...
    "chessboardInnerCornersWidth": 7,
    "chessboardInnerCornersHeight": 4,
...


Run the script as follows: 

python3 autoCalibration.py --fisheye

The script will start capturing images when it detects a valid chessboard pattern every two seconds. It will save the images at: "birds-eye-view/prototype/imgs/calibration", within a directory with the device ID for each camera.

After taking enough valid images, press 'q' on your keyboard to continue with the next camera.

Guidelines:

  • Make sure the chessboard pattern inner corner number matches the configured in the autoCalibrationSettings.json
  • Take around 30-40 pictures for each camera.
  • Use correct valid images for calibration:

The script will show a green border over the image when a chessboard pattern is identified, however, that might not be a good calibration image. Please follow these example images, and try to get many different positions to get a better calibration process.


  • Take some images perpendicular to the camera with the chessboard covering the camera field of view, for example:

C. Top View Image Calibration

The script will automatically continue to this step if the user is calibrating from the fisheye process. Otherwise, start the script with: 

python3 autoCalibration.py

NOTE: Use a high size and resolution chessboard, for example, for the previous fisheye step a calibration size of A4 paper was used, for this step A1 paper will be used.

1. Place the chessboard perpendicular to the camera

2. If the chessboard is valid, press Enter

The script will draw a green border if the chessboard is correctly found. To continue to the next step press Enter key.

3. Review the preview and press Enter

A small preview window with the resulting transformation will be shown. Press Enter key to continue.

4. Repeat the process for each camera

The script will automatically load the next camera according to autoCalibrationSettings.

5. Fine-tune the BEV image

Once all cameras have been configured, the settings can be fine-tuned with controls. For example, you can configure different system layouts if you want to use stitching or not.

Controls can be used to adjust the system as needed:

Refer to the following table for controls:

Table 3. Fine-tuning Script Controls

Key Control Description
Right-click Edit speed Multiplies the factors by 3 to speed up
modification process
H Homography Change to homograpy controls (blue controls)
P Position Change to position controls (green controls)
Left Arrow - FactorX Moves image homography points to the left
Right Arrow + FactorX Moves image homography points to the right
Up Arrow + FactorY Moves up image homography points
Down Arrow - FactorY Moves down image homography points
Q - Scale Zooms out the image
W + Scale Zooms in the image
A - Angle Rotates the image counter clock-wise
S + Angles Rotates the image clock-wise
Z - IPM Reduces IPM view angle
X + IPM Rises IPM view angle


Once the image has been calibrated as desired, press "Enter" key to save the autoCalibrationSettings.json file. If later the calibration needs to be modified but without placing the chessboard for each camera and without fisheye calibration, launch the script as follows: 

python3 autoCalibration.py --load

D. Run the application

Modify the birds-eye-view/src/examples/bevFromCameras.cpp according to your system setup.


1. Select undistort depending on your compatible platform

  • Desktop PC
 auto camFront = factory->makeCamera(CAM_WIDTH,CAM_HEIGHT,CAM_FRAMERATE,"jpegparse",
                                        "jpegdec", "",
                                        "../prototype/autoCalibrationSettings.json",err);
  • NVIDIA Jetson Boards

In Jetson systems, the library uses a gst-cuda compatible plugin to perform fisheye undistortion. 

 auto camFront = factory->makeCamera(CAM_WIDTH,CAM_HEIGHT,CAM_FRAMERATE,"jpegparse",
                                        "jpegdec", "cudaundistort",
                                        "../prototype/autoCalibrationSettings.json",err);

2. Select a valid device ID for each camera 

//Init front camera (Make sure device matches camera position)
err = camFront->load("2");
if(err.isError()) {
    std::cerr << err.getDescription() << std::endl;
    exit(1);
}


3. Compile and run the bevFromCameras Example 

cd birds-eye-view
mkdir build && cd build
meson -Dopencv-cuda=enabled ..
ninja
./src/examples/bevFromCameras

A window with a live feed from cameras with the autoCalibrationSettings will be shown



⇦ Getting_Started Home Getting_Started/Minimal_Application ⇨



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Birds_Eye_View/Getting_Started/Startup_Guide&oldid=58078"