Quick Start Guide
 | WORK IN PROGRESS. Please Contact RidgeRun OR email to support@ridgerun.com if you have any questions. |
| Image Stitching for NVIDIA Jetson |
|---|
|
| Before Starting |
| Image Stitching Basics |
| Overview |
| Getting Started |
| User Guide |
| Resources |
| Examples |
| Spherical Video |
| Performance |
| Contact Us |
Quick Start Guide:
This page describe a brief example on how to use the cuda-stitcher. It should give you a guide of how to stitch images, remind that the Wiki will detail more in each section. To apply this example first build the cuda-stitcher plug-in with its dependencies describe in the Building Image Stitching
Data Set
For this example we will use a set of three images horizontally adjacent. Download the videos located in the following Google Drive folder named Start Guide and save them in a folder named start-guide.
mkdir $HOME/start-guide
The folder must look like:
start-guide ├── video-s0.mp4 ├── video-s1.mp4 └── video-s2.mp4
To calibrate the cameras setup we need to obtain sample images of each sensor and placed them in folders from 0 to N-1 cameras. The calibration tool calculate the homography matrices between the cameras beginning with folder 0 as reference.
We recommend to use the center images as references, so we will put the camera s1 samples in folder 0 to use it as reference. Create the root_folder for the image samples and the indexes folders.
mkdir $HOME/start-guide/image-samples cd $HOME/start-guide/image-samples mkdir 0 1 2
To obtain the samples from the videos use the following pipelines. We would use the ffmpeg method with a framerate of 1/60 to obtain just 6 sample images. And save s1 samples in the folder 0 and s0 samples in the folder 1.
ffmpeg -i $HOME/start-guide/video-s1.mp4 -r 1/60 $HOME/start-guide/image-samples/0/image_s1_%02d.jpg ffmpeg -i $HOME/start-guide/video-s0.mp4 -r 1/60 $HOME/start-guide/image-samples/1/image_s0_%02d.jpg ffmpeg -i $HOME/start-guide/video-s2.mp4 -r 1/60 $HOME/start-guide/image-samples/2/image_s2_%02d.jpg
The image sample folder should look like:
image-samples/
├── 0
│ ├── image_s1_01.jpeg
│ ├── image_s1_02.jpeg
│ ├── image_s1_03.jpeg
│ ├── image_s1_04.jpeg
│ ├── image_s1_05.jpeg
│ └── image_s1_06.jpeg
├── 1
│ ├── image_s0_01.jpeg
│ ├── image_s0_02.jpeg
│ ├── image_s0_03.jpeg
│ ├── image_s0_04.jpeg
│ ├── image_s0_05.jpeg
│ └── image_s0_06.jpeg
└── 2
├── image_s2_01.jpeg
├── image_s2_02.jpeg
├── image_s2_03.jpeg
├── image_s2_04.jpeg
├── image_s2_05.jpeg
└── image_s2_06.jpeg
Now feel free to use the Calibration tool to calculate the homography matrices. Also, you can use the following homography matrices.
{
"homographies": [
{
"images": {
"target": 1,
"reference": 0
},
"matrix": {
"h00": 0.7791246794668455,
"h01": 0.03918377289321621,
"h02": 1017.9359103479511,
"h10": -0.05523859597285526,
"h11": 0.9667746037848984,
"h12": 60.094912688347264,
"h20": -0.00012609656784964485,
"h21": 1.4214840486600683e-05,
"h22": 1.0
}
},
{
"images": {
"target": 2,
"reference": 0
},
"matrix": {
"h00": 1.3913727066002217,
"h01": -0.050979440703839037,
"h02": -1391.5441892163778,
"h10": 0.11215295744983422,
"h11": 1.2533894933660923,
"h12": -191.87470867506212,
"h20": 0.00019946084594618663,
"h21": -3.912626846464952e-05,
"h22": 1.0
}
}
]
}
Save the homography matrices in a JSON file named homographies.json in the path $HOME/start-guide
With the cameras calibrated use the following pipeline in a Jetson platform to create a stitched video of the data set video files. This pipeline took the videos and the homography matrices and perform a warping and blending of the three video files, dumping the result in a mp4 video file output.
Note: Use the videos in the same order the homography matrix was obtain. In this example the order is s1, s0 and s2.
cd $HOME/start-guide/ INPUT_0=video-s1.mp4 INPUT_1=video-s0.mp4 INPUT_2=video-s2.mp4 OUTPUT=stitching.mp4 gst-launch-1.0 -e cudastitcher name=stitcher \ homography-list="`cat homographies.json | tr -d "\n" | tr -d " "`" \ filesrc location=$INPUT_0 ! qtdemux ! h265parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA" ! queue ! stitcher.sink_0 \ filesrc location=$INPUT_1 ! qtdemux ! h265parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA" ! queue ! stitcher.sink_1 \ filesrc location=$INPUT_2 ! qtdemux ! h265parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA" ! queue ! stitcher.sink_2 \ stitcher. ! queue ! nvvidconv ! nvv4l2h265enc bitrate=20000000 ! h265parse ! mp4mux ! filesink location=$OUTPUT
The result should look like the following video:
