Evaluating the Stitcher
| Image Stitching for NVIDIA®Jetson™ |
|---|
|
| Before Starting |
| Image Stitching Basics |
| Overview |
| Getting Started |
| User Guide |
| Resources |
| Examples |
| Spherical Video |
| Performance |
| Contact Us |
Requesting the Evaluation Binary
RidgeRun can provide you with an evaluation binary of the Stitcher to help you test it, in order to request an evaluation binary for a specific architecture, please contact us providing the following information:
- Platform (i.e.: NVIDIA Jetson Orin, TX1/TX2, Xavier, Nano, or x86)
- Jetpack version
- The number of cameras to be used with the stitcher
- The kind of lenses to be used and whether or not you need to apply distortion correction
- Input resolutions and frame rates
- Expected output resolution and frame rate
- Latency requirements
Features of the Evaluation
To help you test our Stitcher, RidgeRun can provide an evaluation version of the plug-in.
The following table summarizes the features available in both the professional and evaluation version of the element.
| Feature | Professional | Evaluation |
|---|---|---|
| Stitcher Examples | Y | Y |
| GstStitcher Element | Y | Y |
| Unlimited Processing Time | Y | N (1) |
| Source Code | Y | N |
(1) The evaluation version will limit the processing to a maximum of 1800 frames.
Evaluating the Stitcher
The stitcher requires the GstCuda plugin in order to work properly, and also depending on your setup, it may require the cuda-undistort element or the rr-projector plug-in, as represented in the diagram below.
RidgeRun provided you with three evaluation tarballs, one for each of the elements mentioned above. The installation should follow that specific order.
Install and test the evaluation binaries
GstCUDA
- If you already own a professional version of this plugin, skip to the cuda undistort element below.
- To install the evaluation binaries follow the instructions found in the GstCuda Installation guide
Cuda Undistort
- This element is only required if the lenses you are using need distortion correction. If that is not the case, skip to the Stitcher element below.
- In order to install the evaluation binaries for this element, follow the Undistort installation guide.
RidgeRun Projector
- This plug-in is only required if the camera's FOV covers 360 degrees. If that is not the case, skip to the Stitcher element below.
- To install the evaluation binaries for this plug-in, follow the Projector installation guide.
Cuda Stitcher
- Install GLib JSON
sudo apt install libjsoncpp-dev sudo apt install libjson-glib-dev
- Once the requirements above are fulfilled, proceed with the installation of the tarball. RidgeRun should've provided you with the following compressed tar package/s:
- cuda-stitcher-X.Y.Z-P-J-eval.tar.gz
- Extract the contents of the file with the following command:
tar xvzf <path-to-evaluation-tar-file>
- The provided cuda-stitcher evaluation version should have the following structure:
cuda-stitcher-X.Y.Z-P-J-eval/ ├── configuration.sh ├── environment.sh ├── examples │ ├── cudafilter_algorithms │ │ ├── gray-scale-filter │ │ │ └── gray-scale-filter.so │ │ ├── median-filter │ │ │ └── median-filter.so │ │ ├── memcpy │ │ │ └── memcpy.so │ │ └── pinhole │ │ └── pinhole.so │ ├── cudamux_algorithms │ │ └── mixer │ │ └── mixer.so │ └── ocvcuda │ ├── stitcher_example │ └── three_image_stitcher ├── include │ ├── gstreamer-1.0 │ │ └── gst │ │ └── stitcher │ │ └── gststitchermaskmeta.h │ └── rr │ └── stitcher │ ├── homography_preprocessor.h │ ├── iblender.h │ ├── icombiner.h │ ├── ihomography.h │ ├── image_geometry_utils.h │ ├── matrix.h │ ├── matrix_utils.h │ ├── ocvcuda │ │ ├── blender.h │ │ └── homography.h │ ├── rgba.h │ ├── runtime_error.h │ └── transformation_node.h ├── lib │ └── aarch64-linux-gnu │ ├── gstreamer-1.0 │ │ ├── libgstcuda.a │ │ ├── libgstcudaexamplefilter.a │ │ ├── libgstcudaexamplefilter.la │ │ ├── libgstcudaexamplefilter.so │ │ ├── libgstcuda.la │ │ ├── libgstcuda.so │ │ └── libgstcudastitcher.so │ ├── libgstcuda-1.0.a │ ├── libgstcuda-1.0.la │ ├── libgstcuda-1.0.so -> libgstcuda-1.0.so.0.100.0 │ ├── libgstcuda-1.0.so.0 -> libgstcuda-1.0.so.0.100.0 │ ├── libgstcuda-1.0.so.0.100.0 │ ├── libgststitcherhelper-1.0.so -> libgststitcherhelper-1.0.so.0 │ ├── libgststitcherhelper-1.0.so.0 -> libgststitcherhelper-X.0.so.Y.Z.0 │ ├── libgststitcherhelper-X.0.so.Y.Z.0 │ ├── librrstitcher-X.Y.Z.so -> librrstitcher-X.Y.Z.so.1 │ ├── librrstitcher-X.Y.Z.so.1 -> librrstitcher-X.Y.Z.so.X.Y.Z │ ├── librrstitcher-X.Y.Z.so.X.Y.Z │ └── pkgconfig │ ├── gst-cuda-1.0.pc │ ├── gststitcherhelper-1.0.pc │ └── rrstitcher-1.0.3.pc └── README
- Then copy the binaries into the standard GStreamer plug-in search path:
sudo cp -r ${PATH_TO_EVALUATION_BINARY}/usr /
- Where
PATH_TO_EVALUATION_BINARYis set to the location in your file system where you have stored the binary provided by RidgeRun (i.e: cuda-stitcher-X.Y.Z-P-J-eval).
- Finally validate that the installation was successful with the following command, you should see the inspect output for the evaluation binary.
gst-inspect-1.0 cudastitcher
How to use the Stitcher
The process of configuring and using the stitcher is explored in depth in the Stitcher user guide.
Testing the Stitcher
You can test the stitcher by running the following example scripts, these perform a stitch between two and three images respectively. It is a great starting point to make sure the installation was successful before moving into more complex examples.
For both examples, you will need to download the example images that will be stitched. You will find the image links in the next sections and an example of how to get them using curl.
Two images example
This example will get two images and stitch them together.
Get the images
 |
 |
| ImageA | Image B |
| https://drive.google.com/uc?export=download&id=15abfY3IM-_3d6Mb6L2EZmYknP5BHhgp_ | https://drive.google.com/uc?export=download&id=1j2-zvXd7Us3nPdQwEB52bcrrNbfQz6Vm |
cd ${PATH_TO_EVALUATION_BINARY}/examples
curl -L -o ImageA.png "https://drive.google.com/uc?export=download&id=15abfY3IM-_3d6Mb6L2EZmYknP5BHhgp_"
curl -L -o ImageB.png "https://drive.google.com/uc?export=download&id=1j2-zvXd7Us3nPdQwEB52bcrrNbfQz6Vm"
Execute the code
cd ${PATH_TO_EVALUATION_BINARY}/examples
./ocvcuda/stitcher_example -f ImageA.png -s ImageB.png
Result
Examine the output image and make sure the stitching was successful, it should look like the following image.
Three images example
This example will get three images and stitch them together.
Get the images
cd ${PATH_TO_EVALUATION_BINARY}/examples
curl -L -o desert_left.png "https://drive.google.com/uc?export=download&id=1XFJu40LweGZgl1zz4ZHpTiUGjiMLSsrF"
curl -L -o desert_center.png "https://drive.google.com/uc?export=download&id=1NS8l0sCCLeJ0ph3hZIwuCbXBlG-WTke6"
curl -L -o desert_right.png "https://drive.google.com/uc?export=download&id=1jlIYOiYGzmrzdXx-o1RC5uKnoxbN8Q4D"
Execute the code
cd ${PATH_TO_EVALUATION_BINARY}/examples
./ocvcuda/three_image_stitcher -l desert_left.png -c desert_center.png -r desert_right.png
Result
Examine the output image and make sure the stitching was successful, it should look like the following image.
Basic validation pipeline
Create a homographies.json file with the following contents:
{
"homographies":[
{
"images":{
"target":0,
"reference":1
},
"matrix":{ "h00": 1.0, "h01": 0.0, "h02": 640.0,
"h10": 0.0, "h11": 1.0, "h12": 0.0,
"h20": 0.0, "h21": 0.0, "h22": 1.0 }
},{
"images":{
"target":2,
"reference":1
},
"matrix":{ "h00": 1.0, "h01": 0.0, "h02": -640.0,
"h10": 0.0, "h11": 1.0, "h12": 0.0,
"h20": 0.0, "h21": 0.0, "h22": 1.0 }
}
]
}
Run the following pipeline, this uses three 640x480 video test sources:
CAPS="video/x-raw(memory:NVMM), width=640, height=480, format=RGBA" HOMOGRAPHY_LIST="`cat homographies.json | tr -d "\n" | tr -d " "`" OUTPUT=output.mp4 gst-launch-1.0 -e \ cudastitcher name=stitcher homography-list=$HOMOGRAPHY_LIST \ videotestsrc pattern=ball num-buffers=30 ! nvvidconv ! $CAPS ! queue ! stitcher.sink_0 \ videotestsrc pattern=smpte num-buffers=30 ! nvvidconv ! $CAPS ! queue ! stitcher.sink_1 \ videotestsrc pattern=snow num-buffers=30 ! nvvidconv ! $CAPS ! queue ! stitcher.sink_2 \ stitcher. ! queue ! nvvidconv ! \ nvv4l2h264enc bitrate=20000000 ! h264parse ! mp4mux ! filesink location=$OUTPUT
For x86 platform, run the following pipeline:
gst-launch-1.0 -e \ cudastitcher name=stitcher homography-list=$HOMOGRAPHY_LIST \ videotestsrc pattern=ball num-buffers=30 ! videoconvert ! $CAPS ! queue ! stitcher.sink_0 \ videotestsrc pattern=smpte num-buffers=30 ! videoconvert ! $CAPS ! queue ! stitcher.sink_1 \ videotestsrc pattern=snow num-buffers=30 ! videoconvert ! $CAPS ! queue ! stitcher.sink_2 \ stitcher. ! queue ! videoconvert ! \ x264enc ! h264parse ! mp4mux ! filesink location=$OUTPUT
The pipeline above generates a 1s 1920x480 output.mp4 video, the first frame should look like the following image.
Running more examples
The Stitcher Examples wiki has many pipeline examples to test the stitcher's functionality; however, make sure to read the Stitcher user guide first.
Troubleshooting
The first level of debugging to troubleshoot a failing evaluation binary is to inspect the GStreamer debug output.
GST_DEBUG=2 gst-launch-1.0
If the output doesn't help you figure out the problem, please contact support@ridgerun.com with the output of the GStreamer debug and any additional information you consider useful.
RidgeRun also offers professional support hours that you can invest in any embedded Linux-related task you want to assign to RidgeRun, such as hardware bring-up tasks, application development, GStreamer pipeline fine-tuning, drivers, etc...