Qualcomm Robotics RB5/RB6 - Software Capture Subsystem

Qualcomm Robotics RB5/RB6
Introduction
SoM Overview Why the RB5/RB6? Carrier Boards Qualcomm Robotics RB5 developer kit Qualcomm Robotics RB6 platform
Software Development Support
Boot Devices Thundercomm SDK Downloading SDK Installing SDK Generating Image and flashing Linaro Getting Started Flashing Bootloader Flashing Boot Image and Rootfs Special / Debug Modes Setting up board in EDL Mode Setting up board in Fastboot Mode
Development in the Board
Getting into the Board Using ADB Using Serial Console Using SSH Device Setup WiFi setup HDMI setup Update Ubuntu Packages
Build from source
Base Image Customize Image
Capture Subsystem
Hardware Capture Components Software Capture Components Building your own camera sensor driver
Image Processing and Acceleration Software
GPU Profiling Fast CV Installing Hexagon SDK Installing Fast CV Building your own Fast CV application
GStreamer Pipelines
Capture and Display Video Encoding Video Decoding
AI Acceleration
Neural Processing SDK Downloading Requirements Setup SDK Environment Install qtimlesnpe Running Inception v3 Running MobileNetV2 + SSD TensorFlow Example pipeline TensorFlow to DLC Conversion
Sensors
IMU
Demos
Drone Demo Smart Camera
Contact Us

This section gives you an overview of the Software APIs available for video capture and custom camera sensor driver development in the Qualcomm Robotics RB5/RB6. The two ways are, the Camera Hal Interface (CHI) image processing driver and the V4L2 API, the latter option is only supported for UVC-compliant cameras.

Camera Hal Interface (CHI)

The CHI API works as a low latency interface between the HAL3 Framework and the camera hardware. CHI offers a flexible fine-grained control over the camera pipeline and access to the several processing engines within the Qualcomm Spectra 480 ISP to allow for customizable use cases. The following are some of the customizable features from the Qualcomm Spectra camera driver you can take advantage from while using CHI:

• CHI override module: An addition ot he HAL3 Interface for explicit image processing pipeline generation, processing engine selection, such as ISP, DSP , and CPU and multi-frame control for any HAL3 camera application.
• CHI pipeline: To construct image processing pipelines consisting of internal camera driver stack Fixed-funtion ISP blocks, or extended nodes, invoked by an external API called native API, which drives features such as, OpenCL and FastCV.
• CHI node extensions: A feature that allows hooks to offload processing on the CPU, GPU or DSP.
• CHI stats overrides: A feature to override Qualcomm's default statistics algorithms, without the need for driver changes.
• CHI Sensor XML: A feature that allows device manufacturers or developers to create drivers, specifying parameters inside structure-defined XML files for specific hardware components in a camera, such as the camera module, the image sensor, actuators, EEPROM and flash.

V4L2 API

Camera devices compliant with USB Device Class (UVC), can be connected via the USB ports available in the Qualcomm Robotics RB5/RB6 mainboard and a driver implementing the V4L2 API is required for them to function, for example, with a camera application that uses standard V4L2 IOCTLs to set properties on the camera or to get video frames.