Birds Eye View - Minimal application of a Birds Eye View application

From RidgeRun Developer Wiki



⇦ Getting_Started/Startup_Guide Home Examples ⇨


Getting Started
User Guide
Calibration Guide
GStreamer
Performance Measurements
Contact Us




Building a BEV Minimal Application

The following example aims to show a minimal full Bird's Eye View library application. It loads frames from 4 input fisheye videos, removes the fisheye distortion, applies the required transformation to each image, computes the BEV, shows the frame to the screen, and saves each frame to a file. If you want to skip code explanation go to full example source code.

Prerequisites

To compile and run the following example, RidgeRun's BEV library must be installed. You can refer to the following Build and Installation Guide.

Code structure

First, include the library headers. The library structure is based on the Factory design pattern. Create a factory object with the desired framework, for example, OpenCV. Then create a loader object, that will be in charge of loading the settings from a JSON file to the settings object as follows: 

#include <bev/bev.h>

int main (int argc, char **argv) {

  bev::runtimeError err;

  //Create factory to build framework objects
  auto factory = bev::iFrameworkFactory::makeFactory(bev::frameworkCode::OPENCV,err);

  //Create settings loader and settings objects
  auto loader = std::make_shared<bev::json::fileSettingsLoader>();
  auto settings = std::make_shared<bev::settings>();

Load the settings from the JSON file. Please notice the way to check if the method ended with an error, for example, if the file was not found. All of the methods in the library that may end with an error can return the error object or use a parameter as a reference to recording the error status of the method. Always checking for the error status is a good practice. The rest of the code in this example does not check for errors in each method that may return an error for simplicity and ease code reading. If you want to review full error checking, you can check the library examples included in the source code. 

  //Load settings from file and verify if loading was correct
  err = loader->load("carSettings.json",settings);
  if(err.isError()){
    std::cerr << err.getDescription() << std::endl;
    exit(1);
  }

Create the Bird's Eye View library engine, which will be in charge of performing the removal of fisheye distortion and computing the BEV. Also, create the needed objects for video display, video saving, and video loading. Also, create an image loader object to load a car image to overlay the BEV output image. 

  //Create Bird's Eye View engine object
  auto engine = factory->makeEngine(settings, err);

  //Create display object
  auto vDisplay = factory->makeDisplay(err);

  //Create videoSaver object
  auto vidSaver = factory->makeVideoSaver(err);
  //Save video to output.avi at 30 FPS
  vidSaver->init("output.avi",settings,30);

  //Create video loaders
  auto vFront = factory->makeVideoLoader(err);
  auto vBack = factory->makeVideoLoader(err);
  auto vLeft = factory->makeVideoLoader(err);
  auto vRight = factory->makeVideoLoader(err);

  //Create image loader for car overlay image
  auto imgLoader = factory->makeImageLoader(err);
  imgLoader->load("../prototype/imgs/docs/topCar.png");
  auto car = imgLoader->getFrame(err);

Load videos from local files, and get the first frame from each of the videos. Here, errors should be checked, for example, if the file was not found, or if the first frame was not valid due to file corruption or any other reason. 

  //Load videos from file
  vFront->load("videos/camera0.mp4");
  vRight->load("videos/camera1.mp4");
  vLeft->load("videos/camera2.mp4");
  vBack->load("videos/camera3.mp4");
  
  //Get first frame from all videos
  auto fFront = vFront->getFrame(err);
  auto fBack = vBack->getFrame(err);
  auto fLeft = vLeft->getFrame(err);
  auto fRight = vRight->getFrame(err);

The following code block is a loop that loads frames from video files while they are available, removes the fisheye distortion, gets a frame with the four images merge in position with the computed BEV perspective. Then optionally a car image overlay can be added using the engine addOverlay method. Also, the current video frame can be saved using the videoSaver object. Finally, the next frame is requested. 

  //Keep running until no more frames are available
  while(fFront != nullptr && fBack != nullptr &&
        fLeft != nullptr && fRight != nullptr ){

    //Remove fisheye distortion from all videos
    fFront = engine->removeFisheye(fFront,0,err);
    fBack = engine->removeFisheye(fBack,3,err);
    fLeft = engine->removeFisheye(fLeft,2,err);
    fRight = engine->removeFisheye(fRight,1,err);
    
    //Compute the BEV with undistorted images
    auto output = engine->getBEV(fFront, fBack, fLeft, fRight, err);

    //Add car overlay to generatedBEV
    output = engine->addOverlay(car,output,err);

    //Show BEV output frame
    vDisplay->showImage(output);
    vDisplay->wait(1);

    //Save BEV frame to output file
    vidSaver->saveFrame(output);

    //Get next frame from videos
    fFront = vFront->getFrame(err);
    fBack = vBack->getFrame(err);
    fLeft = vLeft->getFrame(err);
    fRight = vRight->getFrame(err);

  };
  return 0;
};

Compilation

Basic compilation

To compile the example you must link against the RidgeRun's Bird's Eye View library as follows: 

  g++ example.cpp $(pkg-config --libs --cflags bev) -o example

Then you can execute the example 

  ./example

You will get a live video preview, similar to the following:

Meson compilation

If you are using meson as your build system, you can use the following as a minimal template: 

project('BEV Example', 'cpp',
        version : '0.1.0',
        meson_version : '>= 0.50',
        default_options : ['c_std=c11', 'cpp_std=c++11'])

bev_dep = dependency('bev')

executable('example',
           'example.cpp',
           dependencies: [bev_dep])

Full example source code

/* 
 * Copyright (C) 2019-2020 RidgeRun, LLC (http://www.ridgerun.com)
 * All Rights Reserved.
 *
 * The contents of this software are proprietary and confidential to RidgeRun,
 * LLC.  No part of this program may be photocopied, reproduced or translated
 * into another programming language without prior written consent of
 * RidgeRun, LLC.  The user is free to modify the source code after obtaining
 * a software license from RidgeRun.  All source code changes must be provided
 * back to RidgeRun without any encumbrance. This is a free example of the
 * Bird's Eye View library.
*/

#include <bev/bev.h>

/*
 * Example: Basic example application
 *
 * The following example shows how to use the library to 
 * get a BEV from 4 input fisheye videos, apply an overlay
 * and save it to an output file.
 */
int main (int argc, char **argv) {

  bev::runtimeError err;

  //Create factory to build framework objects
  auto factory = bev::iFrameworkFactory::makeFactory(bev::frameworkCode::OPENCV,err);

  //Create settings loader and settings objects
  auto loader = std::make_shared<bev::json::fileSettingsLoader>();
  auto settings = std::make_shared<bev::settings>();
  
  //Load settings from file and verify if loading was correct
  err = loader->load("carSettings.json",settings);
  if(err.isError()){
    std::cerr << err.getDescription() << std::endl;
    exit(1);
  }

  //Create Bird's Eye View engine object
  auto engine = factory->makeEngine(settings, err);

  //Create display object
  auto vDisplay = factory->makeDisplay(err);

  //Create videoSaver object
  auto vidSaver = factory->makeVideoSaver(err);
  //Save video to output.avi at 30 FPS
  vidSaver->init("output.avi",settings,30);

  //Create video loaders
  auto vFront = factory->makeVideoLoader(err);
  auto vBack = factory->makeVideoLoader(err);
  auto vLeft = factory->makeVideoLoader(err);
  auto vRight = factory->makeVideoLoader(err);

  //Create image loader for car overlay image
  auto imgLoader = factory->makeImageLoader(err);
  imgLoader->load("videos/topCar.png");
  auto car = imgLoader->getFrame(err);

  //Load videos from file
  vFront->load("videos/camera0.mp4");
  vRight->load("videos/camera1.mp4");
  vLeft->load("videos/camera2.mp4");
  vBack->load("videos/camera3.mp4");
  
  //Get first frame from all videos
  auto fFront = vFront->getFrame(err);
  auto fBack = vBack->getFrame(err);
  auto fLeft = vLeft->getFrame(err);
  auto fRight = vRight->getFrame(err);

  //Keep running until no more frames are available
  while(fFront != nullptr && fBack != nullptr &&
        fLeft != nullptr && fRight != nullptr ){

    //Remove fisheye distortion from all videos
    fFront = engine->removeFisheye(fFront,0,err);
    fBack = engine->removeFisheye(fBack,3,err);
    fLeft = engine->removeFisheye(fLeft,2,err);
    fRight = engine->removeFisheye(fRight,1,err);
    
    //Compute the BEV with undistorted images
    auto output = engine->getBEV(fFront, fBack, fLeft, fRight, err);

    //Add car overlay to generatedBEV
    output = engine->addOverlay(car,output,err);

    //Show BEV output frame
    vDisplay->showImage(output);
    vDisplay->wait(1);

    //Save BEV frame to output file
    vidSaver->saveFrame(output);

    //Get next frame from videos
    fFront = vFront->getFrame(err);
    fBack = vBack->getFrame(err);
    fLeft = vLeft->getFrame(err);
    fRight = vRight->getFrame(err);

  };

  return 0;

};



⇦ Getting_Started/Startup_Guide Home Examples ⇨



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Birds_Eye_View/Getting_Started/Minimal_Application&oldid=58073"