NVIDIA Jetson AGX Thor - GstCUDA GStreamer Plug-in for CUDA algorithm integration
GstCUDA Project General Characteristics
GstCUDA characteristics:
- Easy CUDA algorithm integration into GStreamer pipelines.
- Complexity abstraction of both CUDA and GStreamer - allowing the developer to focus on the CUDA algorithm.
- Optimal performance assurance for GStreamer/CUDA applications on Jetson platforms.
Promo Video
Features
- Offers a framework allowing users to develop custom GStreamer elements that can execute any CUDA algorithm.
- Zero memory copy interface between CUDA and GStreamer.
- GstCUDA supports two modes of memory handling:
- NVMM direct mapping mode: use the GstCUDA API's to directly handle NVMM memory buffers. This method provides the best possible performance on the Tegra platforms.
- Unified memory allocator mode: avoids the use of NVMM memory buffers by providing a memory allocator that directly passes the buffer to the GPU, providing zero memory copies and maintaining excellent performance.
- Supports heavy CUDA algorithms and large amounts of data to be processed on the GPU without performance being affected due to copies or memory conversions.
- Provides a set of video filter quick prototyping GStreamer elements, with different input/output combinations, that allows video frames to be processed by the GPU using a custom CUDA library algorithm.
- Provides integrated add-on elements; that consist of a complete shared library that executes a specific CUDA algorithm.
Examples
GstCUDA offers a GStreamer plugin that contains a set of elements, that are ideal for GStreamer/CUDA quick prototyping. The following plugins are used for showcasing GstCUDA capabilities within the Jetson AGX Thor.
- cudafilter
- cudamux
All the examples are using memory type NVMM and the following resolutions and using the performance mode with jetson_clocks.sh
- HD: 1920x1080 with WIDTH=1920 and HEIGHT=1080.
- 4K: 3840X2160 with WIDTH=3840 and HEIGHT=2160.
- FILESRC as the src video.
- FILESINK as the name of the output video.
cudafilter
This element takes the input and then generate a output using the following filters:
- memcpy
- median filter
- grayscale
- pinhole
As an example the grayscale filter generates the following output from the incoming input. The next image shows the after and the before result.
memcpy
This filter copies the specified number of bytes from one memory location to another memory location.
gst-launch-1.0 filesrc location=$FILESRC ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=30/1" ! cudafilter in-place=true location=./memcpy.so ! perf ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILESINK -ev
median filter
This filter removes noise from images, signals, and videos by replacing each pixel's value with the median of its surrounding pixels
gst-launch-1.0 filesrc location=$FILESRC ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=30/1" ! cudafilter in-place=true location=./median-filter.so ! perf ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILESINK -ev
grayscale
This filter provides a grayscale output stream.
gst-launch-1.0 filesrc location=$FILESRC ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=30/1" ! cudafilter in-place=true location=./gray-scale-filter.so ! perf ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILESINK -ev
pinhole
This filter creates an outer edge blur with subtle ghosting / refractions, while maintaining a strong unaffected focal point in the center.
gst-launch-1.0 filesrc location=$FILESRC ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=30/1" ! cudafilter in-place=true location=./pinhole.so ! perf ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILESINK -ev
cudamux
cudamux is a multiple inputs/single output pad video filter GStreamer element, that allows video frames to be processed by the GPU using a custom CUDA library algorithm.
The expected output for a two videotestsrcs using GstCUDA looks like the following image.
The mixer.so CUDA algorithm library consists of a very basic algorithm that receives two YUV I420 images as inputs and mixed them on the GPU, this generates an output image that is the average of the two input images.
gst-launch-1.0 cudamux name=cuda location=./mixer.so filesrc location=$FILESRC1 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=30/1,format=I420" ! cuda.sink_0 filesrc location=$FILESRC2 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=30/1,format=I420" ! queue ! cuda.sink_1 cuda. ! "video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,format=I420,framerate=30/1" ! perf ! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILESINK-ev
Thor Performance
The performance obtained by this element is plotted in the following table for different resolutions. There you can compare the FPS, GPU% and CPU% usage.
| Resolution | GstCUDA's element | CPU (%) | GPU (%) | FPS |
|---|---|---|---|---|
| HD | cudafilter memcpy | 2.473628 | 3.48 | 465.543 |
| cudafilter median filter | 2.74103 | 0.915586 | 281.693 | |
| cudafilter grayscale | 3.911844 | 0.138845 | 434.534 | |
| cudafilter pinhole | 4.129054 | 0.123532 | 479.452 | |
| cudamux mixer | 4.1872895 | 0.292282 | 284.369 | |
| 4K | cudafilter memcpy | 2.44041 | 5.59 | 199.39 |
| cudafilter median filter | 1.528438 | 3.333323 | 120.83 | |
| cudafilter grayscale | 2.474892 | 0.448976 | 199.273 | |
| cudafilter pinhole | 2.3972699 | 0.363429 | 200.485 | |
| cudamux mixer | 3.232201 | 1.057204 | 155.183 |
Getting Started
To know more about the element, please refer to the Features and Limitations wiki page.
How to Purchase
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
