Image Stitching for NVIDIA Jetson - Examples - Nano,TX1,TX2,Xavier Pipelines
Image Stitching for NVIDIA®Jetson™ |
---|
Before Starting |
Image Stitching Basics |
Overview |
Getting Started |
User Guide |
Resources |
Examples |
Spherical Video |
Performance |
Contact Us |
|
This page gives a usage example of the cudastitcher element.
The perf element can be downloaded from this repository, otherwise the element can be removed from the pipeline without any issues. In case of performance issues, consider executing the /usr/bin/jetson_clocks binary.
Example pipelines stitching cameras
Two Camera Stitching
In the two image stitching the left image is always "static" and the transformation is applied on the right image.
Displaying a stitch
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" #TODO
Stitching from two cameras
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" OUTVIDEO=/tmp/stitching_result.ts gst-launch-1.0 -e cudastitcher name=cuda right-center-homography="$HOMOGRAPHY" \ nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 \ nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 \ cuda. ! perf print-arm-load=true ! queue ! nvvidconv ! nvv4l2h264enc bitrate=20000000 ! h264parse ! mpegtsmux ! filesink location=$OUTVIDEO
Generating a MP4 stitch from two cameras
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" OUTVIDEO=/tmp/stitching_result.mp4 gst-launch-1.0 -e cudastitcher name=cuda right-center-homography="$HOMOGRAPHY" \ nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 \ nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 \ cuda. ! perf print-arm-load=true ! queue ! nvvidconv ! nvv4l2h264enc bitrate=20000000 ! h264parse ! mp4mux ! filesink location=$OUTVIDEO
Streaming a stitch from two cameras via UDP+RTP
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" HOST=127.0.0.1 PORT=12345 # Sender gst-launch-1.0 -e cudastitcher name=cuda right-center-homography="$HOMOGRAPHY" \ nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 \ nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 \ cuda. ! perf print-arm-load=true ! nvvidconv ! nvv4l2h264enc ! rtph264pay config-interval=10 ! queue ! udpsink host=$HOST port=$PORT
# Receiver gst-launch-1.0 udpsrc port=$PORT ! 'application/x-rtp, media=(string)video, encoding-name=(string)H264' ! queue ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink
Stitching from three cameras
Displaying a stitch
RC_HOMOGRAPHY="{\"h00\":2.47795806e-01, \"h01\":1.83125651e-01, \"h02\":1511, \"h10\":-9.58951851e-02, \"h11\":1.00466096e+00, \"h12\":3.43046193e+01, \"h20\":-1.84997102e-04, \"h21\":4.48703017e-05, \"h22\":1}" 1962 LC_HOMOGRAPHY="{\"h00\":1.07033034e+00, \"h01\":-8.91500609e-02, \"h02\":-1504, \"h10\":2.06014232e-02, \"h11\":1.06485384e+00, \"h12\":-2.68451772e+01, \"h20\":3.98503995e-06, \"h21\":4.20860985e-05, \"h22\":1}" #TODO
Stitching from two cameras
RC_HOMOGRAPHY="{\"h00\":2.47795806e-01, \"h01\":1.83125651e-01, \"h02\":1511, \"h10\":-9.58951851e-02, \"h11\":1.00466096e+00, \"h12\":3.43046193e+01, \"h20\":-1.84997102e-04, \"h21\":4.48703017e-05, \"h22\":1}" LC_HOMOGRAPHY="{\"h00\":1.07033034e+00, \"h01\":-8.91500609e-02, \"h02\":-1504, \"h10\":2.06014232e-02, \"h11\":1.06485384e+00, \"h12\":-2.68451772e+01, \"h20\":3.98503995e-06, \"h21\":4.20860985e-05, \"h22\":1}" OUTVIDEO=/tmp/stitching_result.ts gst-launch-1.0 -e cudastitcher name=cuda left-center-homography="$LC_HOMOGRAPHY" right-center-homography="$RC_HOMOGRAPHY" \ nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 \ nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 \ nvarguscamerasrc maxperf=true sensor-id=2 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_2 \ cuda. ! perf print-arm-load=true ! queue ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=360" ! nvv4l2h264enc bitrate=20000000 ! h264parse ! mpegtsmux ! filesink location=$OUTVIDEO
Generating a MP4 stitch from two cameras
RC_HOMOGRAPHY="{\"h00\":2.47795806e-01, \"h01\":1.83125651e-01, \"h02\":1511, \"h10\":-9.58951851e-02, \"h11\":1.00466096e+00, \"h12\":3.43046193e+01, \"h20\":-1.84997102e-04, \"h21\":4.48703017e-05, \"h22\":1}" LC_HOMOGRAPHY="{\"h00\":1.07033034e+00, \"h01\":-8.91500609e-02, \"h02\":-1504, \"h10\":2.06014232e-02, \"h11\":1.06485384e+00, \"h12\":-2.68451772e+01, \"h20\":3.98503995e-06, \"h21\":4.20860985e-05, \"h22\":1}" OUTVIDEO=/tmp/stitching_result.mp4 gst-launch-1.0 -e cudastitcher name=cuda left-center-homography="$LC_HOMOGRAPHY" right-center-homography="$RC_HOMOGRAPHY" \ nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 \ nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 \ nvarguscamerasrc maxperf=true sensor-id=2 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_2 \ cuda. ! perf print-arm-load=true ! queue ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=360" ! nvv4l2h264enc bitrate=20000000 ! h264parse ! mp4mux ! filesink location=$OUTVIDEO
Streaming a stitch from two cameras via UDP+RTP
In the three camera case the stitching should always be giving
RC_HOMOGRAPHY="{\"h00\":2.47795806e-01, \"h01\":1.83125651e-01, \"h02\":1511, \"h10\":-9.58951851e-02, \"h11\":1.00466096e+00, \"h12\":3.43046193e+01, \"h20\":-1.84997102e-04, \"h21\":4.48703017e-05, \"h22\":1}" LC_HOMOGRAPHY="{\"h00\":1.07033034e+00, \"h01\":-8.91500609e-02, \"h02\":-1504, \"h10\":2.06014232e-02, \"h11\":1.06485384e+00, \"h12\":-2.68451772e+01, \"h20\":3.98503995e-06, \"h21\":4.20860985e-05, \"h22\":1}" HOST=127.0.0.1 PORT=12345 # Sender gst-launch-1.0 -e cudastitcher name=cuda left-center-homography="$LC_HOMOGRAPHY" right-center-homography="$RC_HOMOGRAPHY" \ nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 \ nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 \ nvarguscamerasrc maxperf=true sensor-id=2 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_2 \ cuda. ! perf print-arm-load=true ! nvvidconv ! nvv4l2h264enc ! rtph264pay config-interval=10 ! queue ! udpsink host=$HOST port=$PORT
# Receiver gst-launch-1.0 udpsrc port=$PORT ! 'application/x-rtp, media=(string)video, encoding-name=(string)H264' ! queue ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink
Example pipeline stitching images
Displaying a stitch form two JPEG images
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" LEFTIMG=left.jpg RIGHTIMG=right.jpg #TODO
Saving a stitch from two JPEG images
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" LEFTIMG=left.jpg RIGHTIMG=right.jpg OUTIMG=/tmp/stitching_result.jpg #TODO
Example pipelines stitching videos
Displaying a stitch from two MP4 videos
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" LEFTVID=left.mp4 RIGHTVID=right.mp4 #TODO
Saving a stitch from two MP4 videos
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" LEFTVID=left.mp4 RIGHTVID=right.mp4 OUTVID=/tmp/stitching_result.mp4 #TODO
Example pipeline for debugging
Dumping output to fakesink
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" gst-launch-1.0 -e cudastitcher name=cuda right-center-homography="$HOMOGRAPHY" nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 cuda. ! perf print-arm-load=true ! fakesink
Generating a raw video frames
HOMOGRAPHY="{\"h00\":7.38511630e-01, \"h01\":1.04317351e-01, \"h02\":1.43471832e+03, \"h10\":-1.07952893e-01, \"h11\":9.89148056e-01, \"h12\":-9.39168804e+00, \"h20\":-2.34496984e-04, \"h21\":3.32061513e-05, \"h22\":1}" OUTVIDEO=/tmp/stitching_result.raw gst-launch-1.0 -e cudastitcher name=cuda right-center-homography="$HOMOGRAPHY" nvarguscamerasrc maxperf=true sensor-id=0 num-buffers=30 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_0 nvarguscamerasrc maxperf=true sensor-id=1 num-buffers=30 ! nvvidconv ! "video/x-raw(memory:NVMM), width=1920, height=1080" ! queue ! cuda.sink_1 cuda. ! perf print-arm-load=true ! filesink location=$OUTVIDEO