Configuration and Calibration
🚧 Documentation is under development
The Video Stitching for Embedded Systems guide is currently under active development. Some sections may be incomplete or change without notice.
Questions? Contact RidgeRun or email to support@ridgerun.com.
Configuration and Calibration
The Stitcher uses a calibration file to describe the geometric relationship between the input cameras. The calibration must correspond to the camera order, input resolution, and physical camera arrangement used by the pipeline.
Existing RidgeRun calibration tools can be used to obtain the homographies, camera distortion parameters, and projection parameters required for the different camera layouts.
Configuration Overview
Both rrstitcher and rrglstitcher use the calibration-file property:
rrstitcher name=stitcher calibration-file=calibration.json
For a rectilinear setup, the calibration contains the homographies that relate the input images to a common reference image.
For N cameras, the calibration normally contains N-1 homographies. Each entry identifies the target and reference camera and contains a 3x3 homography matrix:
homographies ├── target camera index ├── reference camera index └── 3x3 homography matrix
Camera indexes correspond to the Stitcher sink pads:
camera 0 → stitcher.sink_0 camera 1 → stitcher.sink_1 camera 2 → stitcher.sink_2 ...
The sink pads must be connected consecutively from sink_0 to the last camera in the calibration.
A full example of the homography JSON generated by the calibration tool is available in the Image Stitching calibration guide.
Input Requirements and Camera Layout
All Stitcher inputs must use the same resolution and framerate. rrstitcher receives RGBA system-memory buffers, while rrglstitcher receives RGBA GstGLMemory buffers.
The calibration must correspond to the resolution supplied to the Stitcher. Homographies generated for one input resolution should not be used directly with another resolution.
The cameras must have enough overlap for common features to be visible between adjacent views. When collecting samples for calibration, use scenes with good lighting and distinctive objects in the overlap regions.
For a rectilinear camera array, using a center camera as the reference generally provides a convenient calibration layout. The remaining cameras are then related to that reference through the homographies.
For moving scenes, the cameras should be synchronized as closely as possible. This is particularly important when capturing samples for fisheye calibration, where corresponding images should represent the same instant.
Calibration Workflow and Existing Tools
The required calibration depends on the camera and lens type.
For standard rectilinear cameras, the main result needed by the Stitcher is the set of homographies describing the relationship between the camera views. If the lenses introduce noticeable distortion, correct that distortion before estimating the homographies.
Fisheye configurations additionally require projection parameters to convert the fisheye views to a common projection before alignment and blending.
The existing RidgeRun calibration guides cover these steps:
- Image Stitching calibration — estimates the homographies between overlapping camera views and exports them as JSON.
- Camera Undistort Calibration — obtains the camera matrix and distortion parameters used to correct lens distortion.
- Image Projector calibration — obtains the fisheye and equirectangular projection parameters used for 360-degree camera arrangements.
These tools should be used instead of manually estimating the calibration parameters whenever possible.
Rectilinear, Fisheye, and Mask Configuration
Rectilinear
Rectilinear inputs can be stitched directly using the homographies in the calibration file.
If the cameras have significant lens distortion, undistort the inputs before stitching and calculate the homographies using the corrected images.
A typical three-camera rectilinear calibration contains two homographies:
camera 0: reference camera 1: target → camera 0 camera 2: target → camera 0
Camera indexes in the calibration represent the order in which the inputs are connected to the Stitcher.
Fisheye
Fisheye inputs require a projection step in addition to the homography transformation. The fisheye image is converted to an equirectangular representation before the views are aligned and blended.
The Image Projector calibration tool can be used to determine the projection parameters, including:
- Fisheye circle radius and center.
- Lens aperture.
- Camera rotation.
- Horizontal alignment between projected images.
The projected views must still overlap so they can be aligned and blended into the panorama.
Polygon Masks
LibPanorama supports polygon masks to control the regions used during remapping and blending. Masks can be used to exclude parts of an input image or to adjust the blending region between cameras.