360 Videos
Image Stitching for NVIDIA®Jetson™ |
---|
Before Starting |
Image Stitching Basics |
Overview |
Getting Started |
User Guide |
Resources |
Examples |
Spherical Video |
Performance |
Contact Us |
Triple Fisheye Videos
The collection of videos for this setup corresponds to shots in motion where the camera's array was installed over the roof of a car. The shots were divided into clips of 30 seconds or 1 minute to make them manageable and can be found in the dataset folder Triple_Fisheye. Each clip group consists of three MP4 videos, one for each of the cameras, and is saved in its folder named triple_fisheye_vid<index>_<duration>, where an index is a number indicating the clip group id and the duration that can be the 30s or 1m. The following table provides the list of clip groups available for the three fisheye cameras setup:
Scenario | Clip Group | Description |
---|---|---|
Triple fisheye 360 video | triple_fisheye_vid1_30s | Drive down a residential street along a park. |
Triple fisheye 360 video | triple_fisheye_vid2_30s | Turn into a residential street. |
Triple fisheye 360 video | triple_fisheye_vid3_30s | Drive through a roundabout. |
Triple fisheye 360 video | triple_fisheye_vid1_1m | Drive down a residential. |
Each folder contains three video files named as the folder with a suffix indication of the sensor id from s0 to s2.
Triple videos results
Using the 360 Calibration tool the following projection parameters and homography matrix can be obtained.
To watch the stitching result of any of the clip groups, save the above parameters and homography matrix in a JSON file named 360-triple.json
and run the following commands. This pipeline will save the stitching result in a 360_triple_stitched_video.mp4
file.
Dual Fisheye Videos
The collection of videos for this setup consists of shots in motion where the camera's array was installed on top of a box and carried on through a park scenario. The shots were divided into clips with durations of 30 seconds or 1 minute and can be found in the dataset folder Dual_Fisheye. Each clip group of videos consists of MP4 two videos, one for each camera, and saved in a folder named dual_fisheye_vid<index_<duration>, the index indicates the clip group id, and the duration that can be 30s or 1m. The following table provides the list of clips groups available for the dual fisheye cameras setup:
Scenario | Clip Group | Description |
---|---|---|
Dual fisheye 360 video | dual_fisheye_vid1_30s | |
Dual fisheye 360 video | dual_fisheye_vid2_30s | |
Dual fisheye 360 video | dual_fisheye_vid3_30s | |
Dual fisheye 360 video | dual_fisheye_vid1_1m |
Each folder contains two video files named as the folder with a suffix indicating the sensor id s0 or s1.
Dual videos results
Using the 360 Calibration tool the following projection parameters and homography matrix can be obtained.
{ "projections": [ { "0": { "radius": 750.0, "lens": 187.0, "center_x": 993.0, "center_y": 762.0, "rot_x": 0.0, "rot_y": 0.0, "rot_z": -89.6, "fisheye": true } }, { "1": { "radius": 750.3, "lens": 186.0, "center_x": 1044.0, "center_y": 776.0, "rot_x": 0.0, "rot_y": 0.0, "rot_z": 88.7, "fisheye": true } } ], "homographies": [ { "images": { "target": 1, "reference": 0 }, "matrix": { "h00": 1, "h01": 0, "h02": 0.0, "h10": 0, "h11": 1, "h12": 0, "h20": 0, "h21": 0, "h22": 1 } } ] }
To watch the stitching result of any of the clip groups, save the above parameters and homography matrix in a JSON file named 360-dual.json
and run the following commands. This pipeline will save the stitching result in a 360_dual_stitched_video.mp4
file.
S0_C_X=993 S0_C_Y=762 S0_rad=750 S0_R_X=0 S0_R_Y=0 S0_R_Z=-89.6 S0_LENS=187 S1_C_X=1044 S1_C_Y=776 S1_rad=750 S1_R_X=0 S1_R_Y=0 S1_R_Z=88.7 S1_LENS=186 CONFIG_FILE="360-dual.json" HOMOGRAPHIES="`cat $CONFIG_FILE | tr -d "\n" | tr -d "\t" | tr -d " "`" GST_DEBUG=WARNING gst-launch-1.0 -e -v \ cudastitcher name=stitcher homography-list=$HOMOGRAPHIES \ filesrc location=360-s0.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! rreqrprojector center-x=$S0_C_X center-y=$S0_C_Y radius=$S0_rad rot-y=$S0_R_Y rot-z=$S0_R_Z lens=$S0_LENS name=proj0 ! queue ! stitcher.sink_0 \ filesrc location=360-s1.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! rreqrprojector center-x=$S1_C_X center-y=$S1_C_Y radius=$S1_rad rot-y=$S1_R_Y rot-x=$S1_R_Y rot-z=$S1_R_Z lens=$S1_LENS name=proj1 ! queue ! stitcher.sink_1 \ stitcher. ! queue ! nvvidconv ! nvv4l2h264enc bitrate=30000000 ! h264parse ! queue ! qtmux ! filesink location=360_dual_stitched_video.mp4