NVIDIA Jetson AGX Thor - Running Examples
 | The NVIDIA Jetson AGX Thor documentation from RidgeRun is presently being developed. |
Reference Deepstream App
The DeepStream reference application is a GStreamer based solution and consists of set of GStreamer plugins encapsulating low-level APIs to form a complete graph. The reference application has capability to accept input from various sources like camera, RTSP input, encoded file input, and additionally supports multi stream/source capability. In this case we will edit three files to set one, four and thirty frames respectively using tiled display, streanmux, OSD and Primary GIE, you can use the below base code for one frame:
[application] enable-perf-measurement=1 perf-measurement-interval-sec=5 [tiled-display] enable=1 rows=1 columns=1 width=1280 height=720 gpu-id=0 nvbuf-memory-type=0 [source0] enable=1 type=2 uri=file:/opt/nvidia/deepstream/deepstream-8.0/samples/streams/sample_1080p_h264.mp4 num-sources=1 gpu-id=0 cudadec-memtype=0 [sink0] enable=1 type=3 container=1 codec=1 enc-type=0 bitrate=4000000 profile=0 sync=0 output-file=1frame.mp4 source-id=0 gpu-id=0 nvbuf-memory-type=0 [osd] enable=1 gpu-id=0 border-width=1 text-size=15 text-color=1;1;1;1; text-bg-color=0.3;0.3;0.3;1 font=Serif show-clock=0 clock-x-offset=800 clock-y-offset=820 clock-text-size=12 clock-color=1;0;0;0 nvbuf-memory-type=0 [streammux] gpu-id=0 live-source=0 batch-size=1 batched-push-timeout=40000 width=1920 height=1080 enable-padding=0 nvbuf-memory-type=0 [primary-gie] enable=1 gpu-id=0 batch-size=1 bbox-border-color0=1;0;0;1 bbox-border-color1=0;1;1;1 bbox-border-color2=0;0;1;1 bbox-border-color3=0;1;0;1 interval=0 gie-unique-id=1 config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt [tests] file-loop=0
To adapt this base code to four and thirty frames you will need to change the next properties:
- tiled-display
- rows
- columns
- source0
- type
- num-sources
- sink0
- output-file=<change_name>.mp4
The total number of frames (sources) must equal the product of rows × columns defined in the [tiled-display] section.
If you intend to use more than one input source, you must set type=3 for [source0] to enable Multi-URI mode.
How to run it
You can use the next command line to run deepstream reference app:
/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-app/deepstream-app -c /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/<config_file>
Demo result
Once the run is finished, you will see the mp4 output file. Let's see the results for one, four and thirty frames
License Plate detection and recognition
This sample is to show how to use graded models for detection and classification with DeepStream SDK version not less than 5.0.1. The models in this sample are all TAO3.0 models.
Where to get source code
You can get the source code by visiting Deepstream license plate detection and recognition
How to build and run
You will need to install the following requisites:
- Deepstream 6.0 or above, 8.0 preferred for Jetson Thor
Download project using ssh or HTPPS
# SSH
git clone git@github.com:NVIDIA-AI-IOT/deepstream_tao_apps.git
# or HTTPS
git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git
All models can be downloaded with the following commands:
cd deepstream_tao_apps/
sudo ./download_models.sh
Now you can build:
cd apps/tao_others/deepstream_lpr_app
make
Lets run the plate recognition sample:
cp dict_us.txt dict.txt ./deepstream-lpr-app ../../../configs/app/lpr_app_us_config.yml
Known issues and how to fix it
Error: libmosquitto.so.1: cannot open shared object file: No such file or directory.
To fix this issue we will need to install some extra dependencies:
- sudo apt-get install libyaml-cpp-dev libjsoncpp-dev
- sudo apt-get install libmosquitto1
ERROR nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::parseLabelsFile() <nvdsinfer_context_impl.cpp:547> [UID = 1]: Could not open labels file:/home/nvidia/aaguero/deepstream_tao_apps/configs/nvinfer/trafficcamnet_tao/../../../models/trafficcamnet/labels_trafficnet.txt
Search labels_traffic.txt if this file is not under ./deepstream_tao_apps/models/trafficcamnet/ folder you need to search it or create it and copy it into that folder, in order to do that follow the next steps to copy it if exists.
- cd deepstream_tao_apps/
- find . -name "labels_traffic*.txt"
- cp <path to file> ./deepstream_tao_apps/models/trafficcamnet/
or create new one:
- cd deepstream_tao_apps/models/trafficcamnet/
- vi labels_traffic.txt
#write the next lines Car Bicycle Person Roadsign
- save it and run again
ERROR from element file_src_0: Could not open file "/root/deepstream_tao_apps/apps/tao_others/deepstream_lpr_app/us.mp4" for reading. Error details: ../plugins/elements/gstfilesrc.c(557): gst_file_src_start (): /GstPipeline:pipeline/GstFileSrc:file_src_0:
You need to check if you have the correct path to the .mp4 in the .yml file, if not sure please check on:
- vi deepstream_tao_apps/configs/app/lpr_app_us_config.yml
- you need to change the video path on this line:
source-list: list: /opt/nvidia/deepstream/deepstream-8.0/samples/streams/sample_1080p_h264.mp4
Demo result
At the end of the execution, you should see output similar to the following:
End of stream Returned, stopping playback Average fps 30.565279 Totally 105 plates are inferred Deleting pipeline [NvMultiObjectTracker] De-initialized
After the run completes, an output file named out.mp4 will be generated.
Its visual result will look similar to the example below:
Parallel multiple models
The parallel inferencing application constructs the parallel inferencing branches pipeline as the following graph, so that the multiple models can run in parallel in one pipeline.
Where to get source code
You can get the source code by visiting Deepstream parallel multiple models
How to build and run
You will need to install the following requisites:
* /opt/nvidia/deepstream/deepstream/user_additional_install.sh
Download project using ssh or HTPPS
#SSH git clone git@github.com:NVIDIA-AI-IOT/deepstream_reference_apps.git #or HTTPS https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps.git
Now let's download the sample:
cd /deepstream_reference_apps/ apt install git-lfs git lfs install --skip-repo cd deepstream_parallel_inference_app git lfs pull
Let's install dependencies:
apt-get install -y libjson-glib-dev libgstrtspserver-1.0-dev /opt/nvidia/deepstream/deepstream/samples/triton_backend_setup.sh cp tritonclient/sample/gst-plugins/gst-nvdsmetamux/libnvdsgst_metamux.so /opt/nvidia/deepstream/deepstream/lib/gst-plugins/libnvdsgst_metamux.so nvpmodel -m 0 && jetson_clocks cd tritonserver/ ./build_engine.sh
Let's build and run:
cd tritonclient/sample/ source build.sh ./apps/deepstream-parallel-infer/deepstream-parallel-infer -c configs/apps/bodypose_yolo_lpr/source4_1080p_dec_parallel_infer.yml
Demo result
After the run completes, an output file named out.mp4 will be generated. Its visual result will look similar to the example below:
Text recognition with OCD/OCR models
The nvOCDR deepstream sample application for optical character detection and recognition.
Where to get source code
You can get the source code by visiting Text recognition with OCD/OCR models
How to build and run
Download project using ssh or HTPPS
#SSH git clone git@github.com:NVIDIA-AI-IOT/deepstream_reference_apps.git #or HTTPS https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps.git
Now let's set up the environment:
cd /deepstream_tao_apps/apps/tao_others/deepstream-nvocdr-app git lfs pull sudo apt update && sudo apt install -y libopencv-dev git clone https://github.com/NVIDIA-AI-IOT/NVIDIA-Optical-Character-Detection-and-Recognition-Solution.git nvocdr git submodule update --init --recursive make -C nvocdr make -C nvocdr/deepstream CUDA_VER=13.0 cp nvocdr/libnvocdr.so /opt/nvidia/deepstream/deepstream/lib/ cp nvocdr/deepstream/libnvocdr_impl.so /opt/nvidia/deepstream/deepstream/lib/ sudo ldconfig
Build and Run
/usr/src/tensorrt/bin/trtexec --onnx=../../../models/nvocdr/ocdnet.onnx --minShapes=input:1x3x736x1280 --optShapes=input:1x3x736x1280 --maxShapes=input:1x3x736x1280 --fp16 --saveEngine=../../../models/nvocdr/ocdnet.fp16.engine /usr/src/tensorrt/bin/trtexec --onnx=../../../models/nvocdr/ocrnet.onnx --minShapes=input:1x1x64x200 --optShapes=input:32x1x64x200 --maxShapes=input:32x1x64x200 --fp16 --saveEngine=../../../models/nvocdr/ocrnet.fp16.engine # Go to the deepstream-nvocdr-app directory make # Set the 'source-list' as the path of your sources in the nvocdr_app_config.yml ./deepstream-nvocdr-app ../../../configs/app/nvocdr_app_config.yml
Demo result
At the end of each execution, the application generates an output video file (e.g., out.mp4) in the working directory.
This file contains the text recognition results
Object Embedding Vector Generation
The MDX perception sample application drives two Deepstream pipelines, i.e. retail item recognition and people ReID. Retail item recognition pipeline detects retail items from a video and extracts the embedding vector out of every detection bounding box. The embedding vector can form a query to a database of embedding vectors and find the closest match. People ReID detects people from a video and extracts the embedding vector out of every detection bounding box. The pipelines have a primary GIE module detecting the objects of interest from a video frame. The secondary GIE module extracts an embedding vector from the primary GIE result.
How to get the source code
You can get the source code by visiting: deepstream-mdx-perception-app
Build
Download project using ssh or HTPPS
# SSH
git clone git@github.com:NVIDIA-AI-IOT/deepstream_tao_apps.git
# or HTTPS
git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git
All models can be downloaded with the following commands:
cd deepstream_tao_apps/
sudo ./download_models.sh
Let's build the app and run inference one picture:
cd deepstream_tao_apps/apps/tao_others/deepstream-mdx-perception-app make
You can run this application with different configuration files to test various perception models. Each configuration file defines a different DeepStream pipeline and produces unique results.
# Example 1: Run PeopleNet with Re-Identification ./deepstream-mdx-perception-app -c ../../../configs/app/peoplenet_reidentification.yml -m 2 --tiledtext # Example 2: Run Retail Object Detection and Recognition ./deepstream-mdx-perception-app -c ../../../configs/app/retail_object_detection_recognition.yml -m 3 --tiledtext
Known issues
ERROR: Could not open lib: .../post_processor/libnvds_infercustomparser_tao.so NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
Some TAO models required a personalized library, this library usually is under post_processor/ directory inside deepstream_tao_apps so you need to check if that file exist using:
ls ~/aaguero/deepstream_tao_apps/post_processor/ | grep parser if the file exist you will see: libnvds_infercustomparser_tao.so if not you need to compile it so go under /post_processor/ use make CUDA_VER=13.0 command, you can check your CUDA version by running nvcc -version and execute again the app
Demo Result
At the end of each execution, the application generates an output video file (e.g., out.mp4) in the working directory.
This file contains the inference results, such as detected objects, bounding boxes, and re-identification overlays.
Example of output visualization for PeopleNet with Re-Identification:
Example of output visualization for Retail Object Detection and Recognition:
Pose Classification
The project contains pose classification application built using Deepstream SDK.
How to get the source code
You can get the source code by visiting: deepstream-pose-classification
Build
Download project using ssh or HTPPS
# SSH
git clone git@github.com:NVIDIA-AI-IOT/deepstream_tao_apps.git
# or HTTPS
git clone https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps.git
Install libeigen3-dev
sudo apt install libeigen3-dev cd /usr/include sudo ln -sf eigen3/Eigen Eigen
Prepare the models
cd deepstream_tao_apps/
sudo ./download_models.sh
Lets build the application
cd deepstream_tao_apps/apps/tao_others/deepstream-pose-classification/ make CUDA_VER=13.0
Let's run the app:
./deepstream-pose-classification-app ../../../configs/app/deepstream_pose_classification_config.yaml
Known issues
Filesink could not be created. Exiting.
If you cant save the mp4 result, you can edit deepstream_pose_classification_config.yaml
sink: #0 fakesink #1 filesink generate the out.mp4 file in the current directory #2 rtspsink publish at rtsp://localhost:8554/ds-test #3 displaysink sink-type: 1 #encoder type 0=Hardware 1=Software enc-type: 0
In that part of the code you can switch the sink-type option use sink-type: 0 to test the app
Demo result
At the end of the execution, you should see output similar to the following:
End of stream Returned, stopping playback Deleting pipeline objects size:20
After the run completes, an output file named out.mp4 will be generated. Its visual result will look similar to the example below: