Image Stitching for NVIDIA Jetson/Resources/360 data: Difference between revisions
No edit summary |
No edit summary |
||
Line 32: | Line 32: | ||
Each folder contains three video files named as the folder with a suffix indication of the sensor id from s0 to s2. | Each folder contains three video files named as the folder with a suffix indication of the sensor id from s0 to s2. | ||
== Triple videos results == | |||
Using the [[RidgeRun_Image_Projector/User_Guide/Calibration | 360 Calibration tool]] the following projection parameters and homography matrix can be obtained. | |||
<syntaxhighlight lang=bash> | |||
</syntaxhighlight> | |||
To watch the stitching result of any of the clip groups, save the above parameters and homography matrix in a JSON file named <code>360-triple.json</code> and run the following commands. This pipeline will save the stitching result in a <code>360_triple_stitched_video.mp4</code> file. | |||
<syntaxhighlight lang=bash> | |||
</syntaxhighlight> | |||
<!-- | |||
An example stitched video of the triple_fisheye_vid1_30s clip group can be found in the following Google Drive file. | |||
--> | |||
= Dual Fisheye Videos = | = Dual Fisheye Videos = | ||
Line 57: | Line 73: | ||
{{Review|Don't forget the links. We can add them to the table. I will also add the calibration and stitched video |efernandez}} | {{Review|Don't forget the links. We can add them to the table. I will also add the calibration and stitched video |efernandez}} | ||
== Dual videos results == | |||
Using the [[RidgeRun_Image_Projector/User_Guide/Calibration | 360 Calibration tool]] the following projection parameters and homography matrix can be obtained. | |||
<syntaxhighlight lang=bash> | |||
{ | |||
"projections": [ | |||
{ | |||
"0": { | |||
"radius": 750.0, | |||
"lens": 187.0, | |||
"center_x": 993.0, | |||
"center_y": 762.0, | |||
"rot_x": 0.0, | |||
"rot_y": 0.0, | |||
"rot_z": -89.6, | |||
"fisheye": true | |||
} | |||
}, | |||
{ | |||
"1": { | |||
"radius": 750.3, | |||
"lens": 186.0, | |||
"center_x": 1044.0, | |||
"center_y": 776.0, | |||
"rot_x": 0.0, | |||
"rot_y": 0.0, | |||
"rot_z": 88.7, | |||
"fisheye": true | |||
} | |||
} | |||
], | |||
"homographies": [ | |||
{ | |||
"images": { | |||
"target": 1, | |||
"reference": 0 | |||
}, | |||
"matrix": { | |||
"h00": 1, | |||
"h01": 0, | |||
"h02": 0.0, | |||
"h10": 0, | |||
"h11": 1, | |||
"h12": 0, | |||
"h20": 0, | |||
"h21": 0, | |||
"h22": 1 | |||
} | |||
} | |||
] | |||
} | |||
</syntaxhighlight> | |||
To watch the stitching result of any of the clip groups, save the above parameters and homography matrix in a JSON file named <code>360-dual.json</code> and run the following commands. This pipeline will save the stitching result in a <code>360_dual_stitched_video.mp4</code> file. | |||
<syntaxhighlight lang=bash> | |||
S0_C_X=993 | |||
S0_C_Y=762 | |||
S0_rad=750 | |||
S0_R_X=0 | |||
S0_R_Y=0 | |||
S0_R_Z=-89.6 | |||
S0_LENS=187 | |||
S1_C_X=1044 | |||
S1_C_Y=776 | |||
S1_rad=750 | |||
S1_R_X=0 | |||
S1_R_Y=0 | |||
S1_R_Z=88.7 | |||
S1_LENS=186 | |||
CONFIG_FILE="360-dual.json" | |||
HOMOGRAPHIES="`cat $CONFIG_FILE | tr -d "\n" | tr -d " "`" | |||
GST_DEBUG=WARNING gst-launch-1.0 -e -v \ | |||
cudastitcher name=stitcher homography-list=$HOMOGRAPHIES \ | |||
filesrc location=360-s0.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! rreqrprojector center_x=$S0_C_X center_y=$S0_C_Y radius=$S0_rad rot-y=$S0_R_Y rot-z=$S0_R_Z lens=$S0_LENS name=proj0 ! queue ! stitcher.sink_0 \ | |||
filesrc location=360-s1.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! rreqrprojector center_x=$S1_C_X center_y=$S1_C_Y radius=$S1_rad rot-y=$S1_R_Y rot-x=$S1_R_Y rot-z=$S1_R_Z lens=$S1_LENS name=proj1 ! queue ! stitcher.sink_1 \ | |||
stitcher. ! queue ! nvvidconv ! nvv4l2h264enc bitrate=30000000 ! h264parse ! queue ! qtmux ! filesink location=360_dual_stitched_video.mp4 | |||
</syntaxhighlight> | |||
<noinclude> | <noinclude> | ||
{{Image_Stitching_for_NVIDIA_Jetson/Foot|Data Set/Panoramic data|Examples}} | {{Image_Stitching_for_NVIDIA_Jetson/Foot|Data Set/Panoramic data|Examples}} | ||
</noinclude> | </noinclude> |
Revision as of 17:40, 6 October 2023
WORK IN PROGRESS. Please Contact RidgeRun OR email to support@ridgerun.com if you have any questions. |
Image Stitching for NVIDIA®Jetson™ |
---|
Before Starting |
Image Stitching Basics |
Overview |
Getting Started |
User Guide |
Resources |
Examples |
Spherical Video |
Performance |
Contact Us |
Triple Fisheye Videos
The collection of videos for this setup corresponds to shots in motion where the camera's array was installed over the roof of a car. The shots were divided into clips of 30 seconds or 1 minute to make them manageable and can be found in the dataset folder Triple_Fisheye. Each clip group consists of three MP4 videos, one for each of the cameras, and is saved in its own folder named as triple_fisheye_vid<index>_<duration>, where the index is a number indicating the clip group id, and the duration that can be 30s or 1m. The following table provides the list of clips groups available for the three fisheye cameras setup:
Scenario | Clip Group | Description |
---|---|---|
Triple fisheye 360 video | triple_fisheye_vid1_30s | Drive down a residential street along a park. |
Triple fisheye 360 video | triple_fisheye_vid2_30s | Turn into a residential street. |
Triple fisheye 360 video | triple_fisheye_vid3_30s | Drive through a roundabout. |
Triple fisheye 360 video | triple_fisheye_vid1_1m | Drive down a residential. |
Each folder contains three video files named as the folder with a suffix indication of the sensor id from s0 to s2.
Triple videos results
Using the 360 Calibration tool the following projection parameters and homography matrix can be obtained.
To watch the stitching result of any of the clip groups, save the above parameters and homography matrix in a JSON file named 360-triple.json
and run the following commands. This pipeline will save the stitching result in a 360_triple_stitched_video.mp4
file.
Dual Fisheye Videos
The collection of videos for this setup consists of shots in motion where the camera's array was installed on top of a box and carried on through a park scenario. The shots were divided into clips with durations of 30 seconds or 1 minute and can be found in the dataset folder Dual_Fisheye. Each clip group of videos consists of MP4 two videos, one for each camera, and saved in a folder named dual_fisheye_vid<index_<duration>, the index indicates the clip group id, and the duration that can be 30s or 1m. The following table provides the list of clips groups available for the dual fisheye cameras setup:
Scenario | Clip Group | Description |
---|---|---|
Dual fisheye 360 video | dual_fisheye_vid1_30s | |
Dual fisheye 360 video | dual_fisheye_vid2_30s | |
Dual fisheye 360 video | dual_fisheye_vid3_30s | |
Dual fisheye 360 video | dual_fisheye_vid1_1m |
asolis: Missing descriptions (please remove this box when addressed) |
Each folder contains two video files named as the folder with a suffix indicating the sensor id s0 or s1.
efernandez: Don't forget the links. We can add them to the table. I will also add the calibration and stitched video (please remove this box when addressed) |
Dual videos results
Using the 360 Calibration tool the following projection parameters and homography matrix can be obtained.
{ "projections": [ { "0": { "radius": 750.0, "lens": 187.0, "center_x": 993.0, "center_y": 762.0, "rot_x": 0.0, "rot_y": 0.0, "rot_z": -89.6, "fisheye": true } }, { "1": { "radius": 750.3, "lens": 186.0, "center_x": 1044.0, "center_y": 776.0, "rot_x": 0.0, "rot_y": 0.0, "rot_z": 88.7, "fisheye": true } } ], "homographies": [ { "images": { "target": 1, "reference": 0 }, "matrix": { "h00": 1, "h01": 0, "h02": 0.0, "h10": 0, "h11": 1, "h12": 0, "h20": 0, "h21": 0, "h22": 1 } } ] }
To watch the stitching result of any of the clip groups, save the above parameters and homography matrix in a JSON file named 360-dual.json
and run the following commands. This pipeline will save the stitching result in a 360_dual_stitched_video.mp4
file.
S0_C_X=993 S0_C_Y=762 S0_rad=750 S0_R_X=0 S0_R_Y=0 S0_R_Z=-89.6 S0_LENS=187 S1_C_X=1044 S1_C_Y=776 S1_rad=750 S1_R_X=0 S1_R_Y=0 S1_R_Z=88.7 S1_LENS=186 CONFIG_FILE="360-dual.json" HOMOGRAPHIES="`cat $CONFIG_FILE | tr -d "\n" | tr -d " "`" GST_DEBUG=WARNING gst-launch-1.0 -e -v \ cudastitcher name=stitcher homography-list=$HOMOGRAPHIES \ filesrc location=360-s0.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! rreqrprojector center_x=$S0_C_X center_y=$S0_C_Y radius=$S0_rad rot-y=$S0_R_Y rot-z=$S0_R_Z lens=$S0_LENS name=proj0 ! queue ! stitcher.sink_0 \ filesrc location=360-s1.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! queue ! nvvidconv ! rreqrprojector center_x=$S1_C_X center_y=$S1_C_Y radius=$S1_rad rot-y=$S1_R_Y rot-x=$S1_R_Y rot-z=$S1_R_Z lens=$S1_LENS name=proj1 ! queue ! stitcher.sink_1 \ stitcher. ! queue ! nvvidconv ! nvv4l2h264enc bitrate=30000000 ! h264parse ! queue ! qtmux ! filesink location=360_dual_stitched_video.mp4