Image Stitching for NVIDIA Jetson/Spherical Video/Equirectangular Projection: Difference between revisions

Latest revision as of 17:16, 29 September 2023

You need to create an equirectangular projection from each fisheye image. Even though the fisheye only covers a limited angle of view of the real world, the corresponding equirectangular projection image corresponds to the 360 degrees but will have black areas.

The relationship between the fisheye image and the equirectangular projection can be determined geometrically, we are not going to get into details, but you can visualize it in the figures below. As you can see, the region projected in the equirectangular frame will change according to the aperture. For the equirectangular projection of a fisheye of 180 degrees or less half of the image is totally black and doesn’t provide any relevant information, so that memory could be discarded to save resources.

You need to know that a fisheye to equirectangular projector converts the fisheye image where the real-world verticals are curved to actual vertical lines in the equirectangular image and the horizon is converted to an horizontal line. If the resultant equirectangular projection doesn’t look like this you should adjust the projector configuration to accomplish these results.

In a real-world scenario, the fisheye lenses will not be located on the same optical axis and will not have the same characteristics. So to adjust the equirectangular projection to this imperfect setup, you need to define to the projector the fisheye lens characteristics: aperture, radio, and center. Also, you can configure the fisheye rotations to adjust the optical axis, it is assumed that the camera is looking over the Y axis, so the Y rotation works to roll the fisheye lens as if you grab the lens front and rotate your hand. The X-axis is at the side of the camera, so if you change the rotation over this axis you can correct the fisheye tilt angle. And finally, the Z axis is up over the camera and works to pan the fisheye camera over the horizon, this property is rarely used for the 360 stitcher since the stitching stage is in charge of adding the offset to move the equirectangular images horizontally to match them.

RidgeRun implemented a GStreamer element that performs the equirectangular projection from an RGBA image. The element allows the adjustment of the projector setup through properties. The details of the element and how to use it can be found in section X.

Previous: Spherical_Video/About

Index

Next: Spherical_Video/Projector

@@ Line 1: / Line 1: @@
+{{Ambox
+|image=[[File:underconstruction.png|50px]]
+|type=notice
+|issue='''WORK IN PROGRESS. Please [https://www.ridgerun.com/contact <U>Contact</U>] RidgeRun OR email to [mailto:support@ridgerun.com <u>support@ridgerun.com</u>] if you have any questions.'''
+}}
 <noinclude>
-{{Image_Stitching_for_NVIDIA_Jetson/Head|previous=User Guide/Blending|next=Examples|metakeywords=Image Stitching, CUDA, Stitcher, OpenCV, Panorama}}
+{{Image_Stitching_for_NVIDIA_Jetson/Head|previous=Spherical_Video/About|next=Spherical_Video/Projector|metakeywords=Image Stitching, CUDA, Stitcher, OpenCV, Panorama}}
 </noinclude>
 {{DISPLAYTITLE:Equirectangular Projection|noerror}}
+You need to create an equirectangular projection from each fisheye image. Even though the fisheye only covers a limited angle of view of the real world, the corresponding equirectangular projection image corresponds to the 360 degrees but will have black areas.
+The relationship between the fisheye image and the equirectangular projection can be determined geometrically, we are not going to get into details, but you can visualize it in the figures below. As you can see, the region projected in the equirectangular frame will change according to the aperture. For the equirectangular projection of a fisheye of 180 degrees or less half of the image is totally black and doesn’t provide any relevant information, so that memory could be discarded to save resources.
+[[File:Equirectangular-projections.png|400px|thumb|center]]
+You need to know that a fisheye to equirectangular projector converts the fisheye image where the real-world verticals are curved to actual vertical lines in the equirectangular image and the horizon is converted to an horizontal line. If the resultant equirectangular projection doesn’t look like this you should adjust the projector configuration to accomplish these results.
+[[File:Equirectangular-195-overlay.png|800px|thumb|center]]
+In a real-world scenario, the fisheye lenses will not be located on the same optical axis and will not have the same characteristics. So to adjust the equirectangular projection to this imperfect setup, you need to define to the projector the fisheye lens characteristics: aperture, radio, and center. Also, you can configure the fisheye rotations to adjust the optical axis, it is assumed that the camera is looking over the Y axis, so the Y rotation works to roll the fisheye lens as if you grab the lens front and rotate your hand. The X-axis is at the side of the camera, so if you change the rotation over this axis you can correct the fisheye tilt angle. And finally, the Z axis is up over the camera and works to pan the fisheye camera over the horizon, this property is rarely used for the 360 stitcher since the stitching stage is in charge of adding the offset to move the equirectangular images horizontally to match them.
+RidgeRun implemented a GStreamer element that performs the equirectangular projection from an RGBA image. The element allows the adjustment of the projector setup through properties. The details of the element and how to use it can be found in section X.
 <noinclude>
-{{Image_Stitching_for_NVIDIA_Jetson/Foot|User Guide/Blending|Examples}}
+{{Image_Stitching_for_NVIDIA_Jetson/Foot|Spherical_Video/About|Spherical_Video/Projector}}
 </noinclude>