Qualcomm Robotics RB5/Capture Subsystem/Camera Sensor Driver: Difference between revisions

Latest revision as of 13:14, 8 April 2024

The Qualcomm Robotics RB5 Developer Kit include two cameras ready to use for your applications, however, in some cases, you want to use your own camera sensor. In this section you will learn about the files and the building process needed to create a camera sensor driver for the Qualcomm Robotics RB5/RB6 platform. This procedure involves building several images with yocto in a Host PC, that will later be flashed in to the board.

Note

The building process of the images in yocto is very resource intensive. For this we used a host PC with a 14-core 12th Gen Intel(R) Core(TM) i7-12700H processor.

Downloading the LinuxEmbedded source code^[1]

Note

To be able to get the repository you will need to have a Thundercomm account, if you don't already have an account click the following link to create one. Note that Thundercomm will take some time to validate your account.

Note

These instructions are based on the document Thundercomm Turbox™ C5165 Development Kit Linux Enablement Software Release Notes.

The notes are from 2021 and in words of a Thundercomm engineer:

The latest version of LE was released on January 16, 2023. Currently, the release notes of RB5 series has not been released on the official website of Thundercomm, so you need to contact your sales to obtain it separately.

Environment setup

The image is built using a Ubuntu 18.04 container based on an Image provided by Thundercomm.

For this build at least 200GB of disk space is required.

1. Make sure you have docker installed.

2. Download Thundercomm Ubuntu 18.04 image from: Ubuntu-18.04

3. Load the downloaded image with:

docker load < ubuntu-18.04.tar

4. Make sure to authenticate with:

Note

with <User Email> and <password> being your Thundercomm login information.

echo "machine partner.thundercomm.com login <User Email> password <password>" >> ~/.netrc

5. Install repo

sudo apt-get install repo

Get SDK

Once your system is fully configured you can proceed with your build.

1. Create a folder for your SDK:

mkdir SDK_SRC_ROOT
cd SDK_SRC_ROOT

2. Initialize repo

Note

If you file to init the repository make sure your credentials are correct.

repo init -u https://partner.thundercomm.com/sdkgit/a/manifest -b SDK.turbox-rb5-le1.0-dev -m turbox-rb5-le1.0-dev.Pre-FC.r001001.xml

3. Sync repository

repo sync -c -j4 --no-tags

4. Create the docker container for the build.

Note

SDK_SRC_ROOT is the full path in your host where the SDK lives (the folder created in step 1).

Note

ubuntu:18.04 is the docker image you loaded in the environment setup. Be careful if you already had an Ubuntu 18.04 docker image installed when you load Thundercomm's version as there will be a conflict with the names.

sudo docker run -v <SDK_SRC_ROOT>:/home/user/host/ -d --name thundercomm-sdk -p36000:22 ubuntu:18.04

After this step, you will end up with a docker container called thundercomm-sdk with /home/user/host mapped to the host SDK_SRC_ROOT directory.

Start the container

Once you have your docker container ready:

1. Start the container

docker start thundercomm-sdk

2. Log into the container with:

Note

The password of the container is 123456

ssh -p 36000 user@127.0.0.1

Once you are in the container you can enter the host directory where the source code is:

cd host

Note

From this point forward, all the paths specified will be relative to this root directory of the repository, which is /home/user/host.

Camera sensor driver files^[2]

To create a camera driver binary, which will allow access to the camera sensor frames, the CamX framework contains a layer called CHI-CDK with the necessary files and scripts. A camera driver is built upon a series of XML files, shared libraries, and Kernel device tree files, such as the following:

Sensor Software configuration XML

This XML file contains sensor specific information such as power settings, resolution, initialization settings, color filter arrangements and exposure settings. A file with a template showing the complete set of features to specify is found at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor/camxsensordriver.xsd

. The XML file you create for your sensor must be located at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/sensor/<sensor_name>/
<sensor_name>_sensor.xml

, where <sensor_name> is replaced by the name of your sensor, for example, imx577.

Module configuration XML

This XML contains camera module-specific information, including the lens information, mount angles, actuator, OIS, and flash-related information. A file with a template showing the complete set of features to specify is found at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor/camxmoduleconfig.xsd

.

The XML file you create for your sensor module must be located at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/module/<module_name>_module.xml

,

where <module_name> is replaced by the name of your sensor module, for example, cmk_imx577.

Sub-Module configuration XMLs

The camera you are implementing may have additional components, referred to as sub-modules, such as an actuator, EEPROM, Flash and OIS. For these, there are also XML files to describe them. Templates showing the complete set of features to specify for each of them are found at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor/camx<sub_module>driver.xsd

,

where <sub_module> corresponds to one of the submodules, actuator, eeprom, flash or ois.

The XML file you create for your sensor submodule must be located at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/<sub_module>/<sub_module_name>_<sub_module>.xml

,

where <sub_module_name> is replaced by the name of your specific sub-module, for example, ak7371, so an actuator XML file would look like ak7371_actuator.xml.

Sensor gain/exposure customization

The sensor infrastructure provides an alternative for developers and OEMs to create their own gain/exposure functions. To make use of this feature, you will need to create a C/C++ source file that utilizes the C API camxsensordriverAPI.h, which is found at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/api/sensor

.

You will need to generate it as a shared library with the following name com.<vendor_name>.sensor.<sensor_name>.so, for example, com.qti.sensor.imx318.so. You can find an example at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/sensor/imx318

.

Inside the previous directory, the folder

build/linuxembedded

contains a CMakeLists.txt where the shared library creation is specified and the path where it should be installed.

Camera sensor binary build setup

For the system to build the camera driver binary file you need to modify the following files:

1. Include in the device vendor makefile located at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/configs/product.mk

the name of the camera driver binary. Open the file and add the following line:

Note

Replace <module_name> with the same name you specified for the sensor module in the Module configuration XML section, for example cmk_imx577.

PRODUCT_PACKAGES += com.qti.sensormodule.<module_name>.bin

If you created custom gain/exposure functions as in the Sensor gain/exposure customization section. You need to add the name of the library to the product.mk file, as well, for example:

Note

Replace <sensor_name> with the same name you specified for the sensor in the Sensor Software configuration XML section, for example imx577.

PRODUCT_PACKAGES += com.qti.sensor.<sensor_name>

Another step to follow if you created custom gain/exposure functions, is to modify the core CMakeLists's subdirectories:

apps_proc/src/vendor/qcom/proprietary/chi-cdk$ vim core/build/linuxembedded/CMakeLists.txt

You may add a line as follows:

Note

Replace <sensor_name> with the same name you specified for the sensor in the Sensor Software configuration XML section, for example imx577.

add_subdirectory (${CAMX_CHICDK_OEM_PATH}/qcom/sensor/<sensor_name>build/linuxembedded ${CAMX_OUT_PATH}/sensor/<sensor_name>)

2. The infrastructure uses a script to generate the binary file of the camera driver, to locate the paths of the XML files describing the camera, the script reads a yaml file, in which you will need to add the paths for your specific XMLs, open the file

apps_proc/src/vendor/qcom/proprietary/chi-cdk/tools/buildbins/buildbins_kona.yaml

and add the following lines:

Note

Make sure to replace all the placeholders <> with the corresponding names for your sensor. Also note that the sub-modules such as the EEPROM, actuator and flash are not necessary, so if you did not implement them, their respective lines should be removed from the file.

- com.qti.sensormodule.<module_name>:
    - sensor/<sensor_name>/<sensor_name>_sensor.xml
    - module/<module_name>_module.xml
    - eeprom/<sub_module_name>_eeprom.xml
    - actuator/<sub_module_name>_actuator.xml
    - flash/<sub_module_name>_flash.xml

Sensor Device tree configuration

The sensor must be specified in the device tree of your board's kernel by including it in the DTSI files, before the binary can work. The dtsi files corresponding to the RB5/RB6 board is located at

apps_proc/src/kernel/msm-4.19/arch/arm64/boot/dts/vendor/qcom/camera/

. For examples on how to customize the DTSI files, see the files located at

apps_proc/src/kernel/msm-4.19/arch/arm64/boot/dts/vendor/qcom/camera/bindings

.

Building the yocto images and flashing them to the board

Once you are ready with your driver related and device tree changes, you can build the yocto images with the following steps: 1. Build the yocto images:
First, set the right owner on the host folder to avoid permission issues:

chown -R $USER:$USER /home/user/host

Change directory to the host directory and run the building script:

Note

Take into account that with the host PC mentioned at the beginning, the first time you run the script, the process takes around 3.5 hours. In the consequent times that you execute the script after you made a change, for example in the device tree, the process takes around 30 minutes.

cd /home/user/host
./turbox_build.sh

When the images are built you should see a message like the following:

[09:48:14] - update_meta.py: Validating File paths in contents.xml
[09:48:14] - =====================================================
[09:48:14] - update_meta.py: Total number of file paths found in contents.xml = 121
[09:48:14] - update_meta.py: file path validation passed.
[09:48:14] - ============== CHECK COMPLETE ===============
[09:48:14] - update_common_info.py:============ UPDATE COMMON INFO
COMPLETE====================
user@93ed2f966144:~/host$

2. Prepare images for flashing:

./turbox_build.sh --zip_flat_build

This will generate the images for flat installation.

The generated zip file should be under

SDK_SRC_ROOT/turbox/output/

in your host PC, with the following name:

FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001.zip

Flash the image

Once the images are built and packaged you can proceed to the installation:

1. Make sure you have fastboot and adb installed:

sudo apt-get install android-tools-adb fastboot

2. Configure host USB

sudo touch /etc/udev/rules.d/99-android.rules
sudo chmod 777 /etc/udev/rules.d/99-android.rule

Edit /etc/udev/rules.d/99-android.rule and add the following content:

SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0777", GROUP="adm"
SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", MODE="0777", GROUP="adm"

and restart udev:

sudo chmod a+r /etc/udev/rules.d/99-android.rules
sudo service udev restart

3. Unzip the flat build images

unzip FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001.zip

4. Set the board into EDL mode, to learn how to do it see the following section EDL_MODE.

5. Once the board is in EDL mode, enter the directory where the flash script is located and flash the image:

cd FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001/ufs/
sudo ./turbox_flash_flat.sh

If the Flash succeeded you will see something like the following:


10:45:54: INFO:      _             (done)
10:45:54: INFO:     | |                 
10:45:54: INFO:   __| | ___  _ __   ___ 
10:45:54: INFO:  / _` |/ _ \| '_ \ / _ \
10:45:54: INFO: | (_| | (_) | | | |  __/
10:45:54: INFO:  \__,_|\___/|_| |_|\___|
10:45:54: INFO: {All Finished Successfully}

10:45:54: INFO: Overall to target 84.943 seconds (42.93 MBps)
10:45:54: INFO: {percent files transferred 100.00%}

Writing log to '/home/user/FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001/ufs/port_trace.txt', might take a minute


Log is '/home/user/FlatBuild_Turbox-RB5_xx.xx_LE1.0.l.userdebug.Pre-CS.r001001/ufs/port_trace.txt'

6. Reboot the board by pressing the ON/OFF button for 20 seconds. It should boot into your new image.

Rebuilding camera driver binaries

In the process of testing the camera driver you may notice a configuration in an XML file must be modified to meet your requirements or to make the driver functional. In that case, after making the necessary changes in the XMLs, you can rebuild the binary in the source code and copy it back to the

/usr/lib/camera

directory in your board, which simplifies the bring-up process. The steps are the following:

1. To generate the binary for your camera driver go to the directory where the generation script called buildbins3.py is located:

cd apps_proc/src/vendor/qcom/proprietary/chi-cdk/tools/buildbins

2. Run the following command to create the binary:

Note

Take into account this script only generates the binary com.qti.sensormodule.<module_name>.bin.

python3 buildbins3.py --yaml-file-name buildbins_kona.yaml

The binary file will be generated at

apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/bin

directory.

3. To copy the binary file to the board you will need to connect to it via ADB. Click on ADB_GUIDE to learn how to connect via ADB, please skip Step 4 in the mentioned guide.
4. Once the host PC detected the board with adb, run the following commands to give permissions on the board:

adb root
adb disable-verity
adb root
adb remount

5. Change to the directory where the binary file was generated:

cd apps_proc/src/vendor/qcom/proprietary/chi-cdk/oem/qcom/bin

6. Push the binary to the board:

adb push com.qti.sensormodule.<module_name>.bin /usr/lib/camera/

7. Restart the board:

adb reboot

References

↑ Thundercomm. Thundercomm Turbox™ C5165 Development Kit Linux Enablement Software Release Notes. Version 1.2. July 2021
↑ Thundercomm. Thundercomm Turbox™ SOM Camx Architecture Camera Sensor Driver Bringup Guide. Version 1.0. February 2022

⇐

Index

⇒

[releasenotes-1] Thundercomm. Thundercomm Turbox™ C5165 Development Kit Linux Enablement Software Release Notes. Version 1.2. July 2021

[releasenotes_drivers-2] Thundercomm. Thundercomm Turbox™ SOM Camx Architecture Camera Sensor Driver Bringup Guide. Version 1.0. February 2022

[1]

[2]

Latest revision as of 13:14, 8 April 2024

Downloading the LinuxEmbedded source code[1]

Environment setup

Get SDK

Start the container

Camera sensor driver files[2]

Sensor Software configuration XML

Module configuration XML

Sub-Module configuration XMLs

Sensor gain/exposure customization

Camera sensor binary build setup

Sensor Device tree configuration

Building the yocto images and flashing them to the board

Flash the image

Rebuilding camera driver binaries

References

Downloading the LinuxEmbedded source code^[1]

Camera sensor driver files^[2]