About 360 Video

Image Stitching for NVIDIA®Jetson™
Before Starting
Image Stitching Basics
Stitching Projections
Overview
Stitcher Projector Typical Applications Limitations and known issues
Getting Started
Evaluating the Stitcher Getting the code Building Image Stitching
User Guide
Quick Start Guide Controlling the Stitcher Calibration GStreamer Additional details
Resources
3D bases Panoramic data 360 data
Examples
Basic pipelines Using Gstd Other pipelines
Spherical Video
About Equirectangular Projection Projector Building Projector Stitcher Calibration Tool
Performance
Orin Xavier Nano
Contact Us

The RidgeRun’s 360 stitcher is capable of creating a 360 frame from a group of fisheye cameras. The camera’s FOV must cover the 360 region around the cameras, the minimum should be 2 cameras back to back with 180 degrees of FOV.

The resultant 360 frame is nothing else than an equirectangular projection of the world around the cameras, a planar representation of the spherical world coordinates. The most common equirectangular projection in our lives is the world map, where the projection maps the constant spacing meridians to vertical straight lines of constant spacing and the constant spacing circles of latitude to horizontal straight lines of constant spacing.

As you can infer from the world map above, the 360 image consists of 360 degrees of longitude in the horizontal direction and 180 degrees in the vertical. So the 360 stitcher provides an equirectangular image with an aspect ratio of 2:1. As shown in the following image:

In the 360 image/equirectangular panoramic image all the verticals remain vertical and the horizon becomes a straight line across the middle of the image. Also the coordinates in the image relate linearly to pan and tilt angles in the real world. Making the poles to be located at the top and bottom sides of the image, stretched to the entire image width, and the areas near the poles get stretched horizontally.

In the case of the cameras, each camera's equirectangular projection would generate a portion of the 360 image projection depending on the lenses’ specifications, the equirectangular projection section below will get into more details about it.

In general, the 360 stitcher full process follows the next steps: capture from an array of fisheye cameras that covers the 360 degrees longitude, projection of each fisheye image to an equirectangular image and stitching of the equirectangular projections to create a blended 360 image.

Fisheye Lenses

For 360 stitching so far only cameras with fisheye lenses are supported. Fisheye lens is a wide-angle lens with a distortion that allows a small number of cameras to cover the 360 world around the cameras. The fisheye cameras produce images with a circular scenario and black borders as shown below.

However, depending on the sensor that you are using and the capture configuration, you may receive fisheye images cropped as shown below, having just a partial section of the fisheye circle. The 360 stitcher will be able to handle those types of images as well, just need to set the appropriate parameters (described later) and remember that your final 360 frame will have missing information around the poles.

For the stitcher, the fisheyes images are going to be considered as circular fisheyes where the longitude is radially symmetrical and the latitude proportional to the radius from the center of the fisheye circle. Most lenses will have a distortion that differs from the previous ideal description but for now, that distortion will be ignored.

The most important features of the fisheye image for the stitcher are: aperture, radius and center.

• Aperture: indicates the angle of view of the fisheye lens. Fisheye lenses can capture 180 degrees and up.
• Radius: is the radius of the fisheye circle that matches the aperture.
• Center: is the center of the fisheye circle