This page describe a brief example on how to use the cuda-stitcher. It should give you a guide of how to stitch images, remind that the Wiki will detail more in each section. To apply this example first build the cuda-stitcher plug-in with its dependencies describe in the Building Image Stitching

Data Set

For this example we will use a set of three images horizontally adjacent. Download the videos located in the following Google Drive folder named Start Guide and save them in a folder named start-guide.

mkdir $HOME/start-guide

The folder must look like:

start-guide
├── video-s0.mp4
├── video-s1.mp4
└── video-s2.mp4

To calibrate the cameras setup we need to obtain sample images of each sensor and placed them in folders from 0 to N-1 cameras. The calibration tool calculate the homography matrices between the cameras beginning with folder 0 as reference.

We recommend to use the center images as references, so we will put the camera s1 samples in folder 0 to use it as reference. Create the root_folder for the image samples and the indexes folders.

mkdir $HOME/start-guide/image-samples
cd $HOME/start-guide/image-samples
mkdir 0 1 2

To obtain the samples from the videos use the following pipelines. We would use the ffmpeg method with a framerate of 1/60 to obtain just 6 sample images. And save s1 samples in the folder 0 and s0 samples in the folder 1.

ffmpeg -i $HOME/start-guide/video-s1.mp4 -r 1/60 $HOME/start-guide/image-samples/0/image_s1_%02d.jpg
ffmpeg -i $HOME/start-guide/video-s0.mp4 -r 1/60 $HOME/start-guide/image-samples/1/image_s0_%02d.jpg
ffmpeg -i $HOME/start-guide/video-s2.mp4 -r 1/60 $HOME/start-guide/image-samples/2/image_s2_%02d.jpg

The image sample folder should look like:

image-samples/
├── 0
│   ├── image_s1_01.jpeg
│   ├── image_s1_02.jpeg
│   ├── image_s1_03.jpeg
│   ├── image_s1_04.jpeg
│   ├── image_s1_05.jpeg
│   └── image_s1_06.jpeg
├── 1
│   ├── image_s0_01.jpeg
│   ├── image_s0_02.jpeg
│   ├── image_s0_03.jpeg
│   ├── image_s0_04.jpeg
│   ├── image_s0_05.jpeg
│   └── image_s0_06.jpeg
└── 2
    ├── image_s2_01.jpeg
    ├── image_s2_02.jpeg
    ├── image_s2_03.jpeg
    ├── image_s2_04.jpeg
    ├── image_s2_05.jpeg
    └── image_s2_06.jpeg

Now feel free to use the Calibration tool to calculate the homography matrices. Also, you can use the following homography matrices.

{
    "homographies": [
        {
            "images": {
                "target": 1,
                "reference": 0
            },
            "matrix": {
                "h00": 0.7791246794668455,
                "h01": 0.03918377289321621,
                "h02": 1017.9359103479511,
                "h10": -0.05523859597285526,
                "h11": 0.9667746037848984,
                "h12": 60.094912688347264,
                "h20": -0.00012609656784964485,
                "h21": 1.4214840486600683e-05,
                "h22": 1.0
            }
        },
        {
            "images": {
                "target": 2,
                "reference": 0
            },
            "matrix": {
                "h00": 1.3913727066002217,
                "h01": -0.050979440703839037,
                "h02": -1391.5441892163778,
                "h10": 0.11215295744983422,
                "h11": 1.2533894933660923,
                "h12": -191.87470867506212,
                "h20": 0.00019946084594618663,
                "h21": -3.912626846464952e-05,
                "h22": 1.0
            }
        }
    ]
}

Save the homography matrices in a JSON file named homographies.json in the path $HOME/start-guide

With the cameras calibrated use the following pipeline in a Jetson platform to create a stitched video of the data set video files. This pipeline took the videos and the homography matrices and perform a warping and blending of the three video files, dumping the result in a mp4 video file output.

Note: Use the videos in the same order the homography matrix was obtain. In this example the order is s1, s0 and s2.

cd $HOME/start-guide/

INPUT_0=video-s1.mp4
INPUT_1=video-s0.mp4
INPUT_2=video-s2.mp4

OUTPUT=stitching.mp4

gst-launch-1.0 -e cudastitcher name=stitcher \
homography-list="`cat homographies.json | tr -d "\n" | tr -d " "`" \
  filesrc location=$INPUT_0 ! qtdemux ! h265parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA" ! queue ! stitcher.sink_0 \
  filesrc location=$INPUT_1 ! qtdemux ! h265parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA" ! queue ! stitcher.sink_1 \
  filesrc location=$INPUT_2 ! qtdemux ! h265parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=RGBA" ! queue ! stitcher.sink_2 \
  stitcher. ! queue ! nvvidconv ! nvv4l2h265enc bitrate=20000000 ! h265parse ! mp4mux ! filesink location=$OUTPUT

The result should look like the following video:

Previous: User Guide

Index

Next: User Guide/Controlling the Stitcher

❯