Qualcomm Robotics RB5/RB6 - Building your own camera sensor driver
The Qualcomm Robotics RB5 Developer Kit include two cameras ready to use for your applications, however, in some cases, you want to use your own camera sensor. In this section you will learn about the files and the building process needed to create a camera sensor driver for the Qualcomm Robotics RB5/RB6 platform. This procedure involves building several images with yocto in a Host PC, that will later be flashed in to the board.
Downloading the LinuxEmbedded source code[1]
The notes are from 2021 and in words of a Thundercomm engineer:
The latest version of LE was released on January 16, 2023. Currently, the release notes of RB5 series has not been released on the official website of Thundercomm, so you need to contact your sales to obtain it separately.
Environment setup
The image is built using a Ubuntu 18.04 container based on an Image provided by Thundercomm.
For this build at least 200GB of disk space is required.
1. Make sure you have docker installed.
2. Download Thundercomm Ubuntu 18.04 image from: Ubuntu-18.04
3. Load the downloaded image with:
docker load < ubuntu-18.04.tar
4. Make sure to authenticate with:
echo "machine partner.thundercomm.com login <User Email> password <password>" >> ~/.netrc
5. Install repo
sudo apt-get install repo
Get SDK
Once your system is fully configured you can proceed with your build.
1. Create a folder for your SDK:
mkdir SDK_SRC_ROOT cd SDK_SRC_ROOT
2. Initialize repo
repo init -u https://partner.thundercomm.com/sdkgit/a/manifest -b SDK.turbox-rb5-le1.0-dev -m turbox-rb5-le1.0-dev.Pre-FC.r001001.xml
3. Sync repository
repo sync -c -j4 --no-tags
4. Create the docker container for the build.
sudo docker run -v <SDK_SRC_ROOT>:/home/user/host/ -d --name thundercomm-sdk -p36000:22 ubuntu:18.04
After this step, you will end up with a docker container called thundercomm-sdk with /home/user/host mapped to the host SDK_SRC_ROOT directory.
Start the container
Once you have your docker container ready:
1. Start the container
docker start thundercomm-sdk
2. Log into the container with:
ssh -p 36000 user@127.0.0.1
Once you are in the container you can enter the host directory where the source code is:
cd host
Camera sensor driver files[2]
To create a camera driver binary, which will allow access to the camera sensor frames, the CamX framework contains a layer called CHI-CDK with the necessary files and scripts. A camera driver is built upon a series of XML files, shared libraries, and Kernel device tree files, such as the following:
Sensor Software configuration XML
This XML file contains sensor specific information such as power settings, resolution, initialization settings, color filter arrangements and exposure settings. A file with a template showing the complete set of features to specify is found at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor/camxsensordriver.xsd
. The XML file you create for your sensor must be located at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/sensor/<sensor_name>/ <sensor_name>_sensor.xml
, where <sensor_name> is replaced by the name of your sensor, for example, imx577.
Module configuration XML
This XML contains camera module-specific information, including the lens information, mount angles, actuator, OIS, and flash-related information. A file with a template showing the complete set of features to specify is found at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor/camxmoduleconfig.xsd
.
The XML file you create for your sensor module must be located at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/module/<module_name>_module.xml
,
where <module_name> is replaced by the name of your sensor module, for example, cmk_imx577.
Sub-Module configuration XMLs
The camera you are implementing may have additional components, referred to as sub-modules, such as an actuator, EEPROM, Flash and OIS. For these, there are also XML files to describe them. Templates showing the complete set of features to specify for each of them are found at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor/camx<sub_module>driver.xsd
,
where <sub_module> corresponds to one of the submodules, actuator, eeprom, flash or ois.
The XML file you create for your sensor submodule must be located at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/<sub_module>/<sub_module_name>_<sub_module>.xml
,
where <sub_module_name> is replaced by the name of your specific sub-module, for example, ak7371, so an actuator XML file would look like ak7371_actuator.xml.
Sensor gain/exposure customization
The sensor infrastructure provides an alternative for developers and OEMs to create their own gain/exposure functions. To make use of this feature, you will need to create a C/C++ source file that utilizes the C API camxsensordriverAPI.h, which is found at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor
.
You will need to generate it as a shared library with the following name com.<vendor_name>.sensor.<sensor_name>.so, for example, com.qti.sensor.imx318.so. You can find an example at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/sensor/imx318
.
Inside the previous directory, the folder
build/linuxembedded
contains a CMakeLists.txt where the shared library creation is specified and the path where it should be installed.
Camera sensor binary build setup
For the system to build the camera driver binary file you need to modify the following files:
1. Include in the device vendor makefile located at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/configs/product.mk
the name of the camera driver binary. Open the file and add the following line:
PRODUCT_PACKAGES += com.qti.sensormodule.<module_name>.bin
If you created custom gain/exposure functions as in the Sensor gain/exposure customization section. You need to add the name of the library to the product.mk file, as well, for example:
PRODUCT_PACKAGES += com.qti.sensor.<sensor_name>
Another step to follow if you created custom gain/exposure functions, is to modify the core CMakeLists's subdirectories:
apps_proc/src/vendor/qcom/proprietary/chi-cdk$ vim core/build/linuxembedded/CMakeLists.txt
You may add a line as follows:
add_subdirectory (${CAMX_CHICDK_OEM_PATH}/qcom/sensor/<sensor_name>build/linuxembedded ${CAMX_OUT_PATH}/sensor/<sensor_name>)
2. The infrastructure uses a script to generate the binary file of the camera driver, to locate the paths of the XML files describing the camera, the script reads a yaml file, in which you will need to add the paths for your specific XMLs, open the file
apps_proc/src/vendor/qcom/proprietary/chi-cdk/tools/buildbins/buildbins_kona.yaml
and add the following lines:
- com.qti.sensormodule.<module_name>:
- sensor/<sensor_name>/<sensor_name>_sensor.xml
- module/<module_name>_module.xml
- eeprom/<sub_module_name>_eeprom.xml
- actuator/<sub_module_name>_actuator.xml
- flash/<sub_module_name>_flash.xml
Sensor Device tree configuration
The sensor must be specified in the device tree of your board's kernel by including it in the DTSI files, before the binary can work. The dtsi files corresponding to the RB5/RB6 board is located at
apps_proc/src/kernel/msm-4.19/arch/arm64/boot/dts/vendor/qcom/camera/
. For examples on how to customize the DTSI files, see the files located at
apps_proc/src/kernel/msm-4.19/arch/arm64/boot/dts/vendor/qcom/camera/bindings
.
Building the yocto images and flashing them to the board
Once you are ready with your driver related and device tree changes, you can build the yocto images with the following steps:
1. Build the yocto images:
First, set the right owner on the host folder to avoid permission issues:
chown -R $USER:$USER /home/user/host
Change directory to the host directory and run the building script:
cd /home/user/host ./turbox_build.sh
When the images are built you should see a message like the following:
[09:48:14] - update_meta.py: Validating File paths in contents.xml [09:48:14] - ===================================================== [09:48:14] - update_meta.py: Total number of file paths found in contents.xml = 121 [09:48:14] - update_meta.py: file path validation passed. [09:48:14] - ============== CHECK COMPLETE =============== [09:48:14] - update_common_info.py:============ UPDATE COMMON INFO COMPLETE==================== user@93ed2f966144:~/host$
2. Prepare images for flashing:
./turbox_build.sh --zip_flat_build
This will generate the images for flat installation.
The generated zip file should be under
SDK_SRC_ROOT/turbox/output/
in your host PC, with the following name:
FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001.zip
Flash the image
Once the images are built and packaged you can proceed to the installation:
1. Make sure you have fastboot and adb installed:
sudo apt-get install android-tools-adb fastboot
2. Configure host USB
sudo touch /etc/udev/rules.d/99-android.rules sudo chmod 777 /etc/udev/rules.d/99-android.rule
Edit /etc/udev/rules.d/99-android.rule and add the following content:
SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0777", GROUP="adm"
SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", MODE="0777", GROUP="adm"
and restart udev:
sudo chmod a+r /etc/udev/rules.d/99-android.rules sudo service udev restart
3. Unzip the flat build images
unzip FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001.zip
4. Set the board into EDL mode, to learn how to do it see the following section EDL_MODE.
5. Once the board is in EDL mode, enter the directory where the flash script is located and flash the image:
cd FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001/ufs/ sudo ./turbox_flash_flat.sh
If the Flash succeeded you will see something like the following:
10:45:54: INFO: _ (done)
10:45:54: INFO: | |
10:45:54: INFO: __| | ___ _ __ ___
10:45:54: INFO: / _` |/ _ \| '_ \ / _ \
10:45:54: INFO: | (_| | (_) | | | | __/
10:45:54: INFO: \__,_|\___/|_| |_|\___|
10:45:54: INFO: {All Finished Successfully}
10:45:54: INFO: Overall to target 84.943 seconds (42.93 MBps)
10:45:54: INFO: {percent files transferred 100.00%}
Writing log to '/home/user/FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001/ufs/port_trace.txt', might take a minute
Log is '/home/user/FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001/ufs/port_trace.txt'
6. Reboot the board by pressing the ON/OFF button for 20 seconds. It should boot into your new image.
Rebuilding camera driver binaries
In the process of testing the camera driver you may notice a configuration in an XML file must be modified to meet your requirements or to make the driver functional. In that case, after making the necessary changes in the XMLs, you can rebuild the binary in the source code and copy it back to the
/usr/lib/camera
directory in your board, which simplifies the bring-up process. The steps are the following:
1. To generate the binary for your camera driver go to the directory where the generation script called buildbins3.py is located:
cd apps_proc/src/vendor/qcom/proprietary/chi-cdk/tools/buildbins
2. Run the following command to create the binary:
python3 buildbins3.py --yaml-file-name buildbins_kona.yaml
The binary file will be generated at
apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/bin
directory.
3. To copy the binary file to the board you will need to connect to it via ADB. Click on ADB_GUIDE to learn how to connect via ADB, please skip Step 4 in the mentioned guide.
4. Once the host PC detected the board with adb, run the following commands to give permissions on the board:
adb root adb disable-verity adb root adb remount
5. Change to the directory where the binary file was generated:
cd apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/bin
6. Push the binary to the board:
adb push com.qti.sensormodule.<module_name>.bin /usr/lib/camera/
7. Restart the board:
adb reboot
References