Qualcomm Robotics RB5/RB6 - Neural Processing SDK: Install qtimlenspe
In this wiki, we will see the steps to build and install the qtimlenspe GStreamer element in the board[1]. This is necessary since in the latest LU image; as of March 15, 2023, the element does not come installed. From the section Downloading Requirements and Setup SDK, we already downloaded and set up the Neural Processing SDK version 1.43.0. Please, follow the next steps to get the element:
1. Connect your board to your host computer through ADB. You can check how to do it in our ADB section.
2. Open a terminal and move to where you downloaded and unzipped your Neural Processing SDK.
cd /path/to/SDK/snpe-1.43.0.2307
3. Now, we will move some libraries from the SDK in our host PC to the Qualcomm Robotics RB5/RB6 board. These libraries are the skeleton needed to build our element.
adb push lib/aarch64-ubuntu-gcc7.5/lib* /usr/lib/ adb push lib/aarch64-ubuntu-gcc7.5/libsnpe_dsp_domains_v2.so /usr/lib/rfsa/adsp/ adb push lib/dsp/libsnpe_dsp_v66_domains_v2_skel.so /usr/lib/rfsa/adsp/
4. Enter the board via adb shell.
adb shell
5. Install cmake. This is the tool we are using for creating and compiling our element.
apt install cmake
6. Create a directory to save our files for development.
mkdir -p ~/DEVELOPMENT/ && cd ~/DEVELOPMENT
7. Clone the repository with the source files of the qtimlesnpe element
git clone https://github.com/quic/sample-apps-for-robotics-platforms.git
8. Move to the directoy with the source files for the SNPE element.
cd sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe
9. Create a new folder, where we will include libraries from the Neural Processing SDK.
mkdir snpe-sdk && cd snpe-sdk
Inside it, create a new lib folder to store the libraries from lib/aarch64-ubuntu-gcc7.5/:
mkdir lib
10. Now, without closing the current terminal, please open another one in your host computer. We will use this new terminal to move files from the Neural Processing SDK to the board. In the new terminal, first move to where you downloaded and unzipped the SDK.
cd /path/to/SDK/snpe-1.43.0.2307
Then, push include folder of the SDK that contains every needed header.
adb push include/ ~/DEVELOPMENT/sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe/snpe-sdk
Finally, push all the libraries from lib/aarch64-ubuntu-gcc7.5/ to the lib directory previously created.
adb push lib/aarch64-ubuntu-gcc7.5/lib* ~/DEVELOPMENT/sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe/snpe-sdk/lib/
You can close the new terminal opened in your host computer. Do not close the terminal opened with adb shell
11. Moving back to the board, with the terminal opened with adb shell, you can check if everything went well if your top level structure looks like this:
~/DEVELOPMENT/sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe
├── CMakeLists.txt
├── README.txt
├── config.h.in
├── mle_engine
│ ├── CMakeLists.txt
│ ├── *
│ └── snpe_single_ssd.h
├── mle_gst_snpe
│ ├── CMakeLists.txt
│ ├── *
│ └── mle_snpe.h
└── snpe-sdk
├── include
│ └── zdl
│ ├── DiagLog
│ │ ├── IDiagLog.hpp
│ │ └── *
│ ├── DlContainer
│ │ └── IDlContainer.hpp
│ ├── DlSystem
│ │ ├── DlEnums.hpp
│ │ ├── *
│ │ └── ZdlExportDefine.hpp
│ ├── PlatformValidator
│ │ └── PlatformValidator.hpp
│ ├── SNPE
│ │ ├── ApplicationBufferMap.hpp
│ │ ├── *
│ │ └── UserBufferList.hpp
│ └── SnpeUdo
│ ├── UdoBase.h
│ ├── *
│ └── UdoShared.h
└── lib
├── libSNPE.so
├── *
└── libsnpe_dsp_domains_v2.so
12. While inside the directory ~/DEVELOPMENT/sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe, create a new build directory to store every file of the compilation process.
mkdir build && cd build
13. Use cmake to generate the build files of the element.
cmake -DSNPE_SDK_BASE_DIR=~/DEVELOPMENT/sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe/snpe-sdk ..
The output of the above command should be similar to the following:
-- Configuring done -- Generating done -- Build files have been written to: ~/DEVELOPMENT/sample-apps-for-robotics-platforms/gst_mle_oss/gst_snpe/build
14. Now, you can compile the element with the Makefile file generated.
make
The output of the above command should be similar to the following:
Scanning dependencies of target gstqtimlesnpe [ 87%] Building CXX object mle_gst_snpe/CMakeFiles/gstqtimlesnpe.dir/mle_snpe.cc.o [100%] Linking CXX shared library libgstqtimlesnpe.so [100%] Built target gstqtimlesnpe
15. Now, install the element in the board.
make install
The output of the above command should be similar to the following:
Install the project... -- Install configuration: "Release" -- Installing: /usr/lib/libEngine_MLE_SNPE.so -- Set runtime path of "/usr/lib/libEngine_MLE_SNPE.so" to "" -- Installing: /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstqtimlesnpe.so -- Set runtime path of "/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstqtimlesnpe.so" to ""
16. Finally, you can check if everything went fine by checking the elements of GStreamer with the following command:
gst-inspect-1.0 | grep qti
The output of the above command should be similar to the following:
qtimlesnpe: qtimlesnpe: MLE SNPE
17. Now you have a new GStreamer element ready to be used with your ML models! Please check our next section to view an example project using the SDK and the qtimlesnpe element.
References