Porting a new camera sensor to the Toradex Verdin i.MX95 Platform
Overview
The following guide covers the basics for porting a new camera sensor for the i.MX95 with Toradex Verdin Mallow carrier. The i.MX95 uses LibCamera, a complete framework for image capturing and processing.
Camera sensor driver
NXP has a guide to port new sensors to the i.MX95 board. The new driver can follow the common linux kernel driver structure.
- The sensor is controlled using i2c, so the driver needs to provide functions to read and write the i2c interface.
- IOCTL definitions for common subdevice operations like core, pad and video.
- Subdevice controls for exposure, gain and others.
- Driver registration routine.
- Sensor mode lookup tables and initialization sequence.
According to NXP sensor porting guide, the driver must implement at least controls for exposure, gain, link frequency and pixel rate. Some additional considerations:
- LibCamera will control when to start or stop streaming, it is recommended to initialize the sensor with streaming paused - stopped.
- Consider clock and power management only if the sensor module does not provide them. Check your hardware configuration to make sure the sensor receives the expected clock and power from either the i.MX95 or a sensor adapter.
Device Tree changes
Device tree will depend on the sensor used, and whether it has an adapter or not. A sensor module adapter can provide signals like clock and power to the sensor module, which simplifies the device tree.
- If the sensor adapter provides clock, it is recommended to use a "fixed-clock" node in the device tree to describe sensor clock settings like frequency.
- If the sensor adapter does not provide clock, then the board must provide it. Make sure you configure the device tree accordingly.
- The reset pin of the CSI port in the Verdin Mallow carrier is mapped to the GPIO 4, line 28. Make sure you check if the sensor uses the reset signal in low or high state.
- The i2c bus controller where the sensor will be connected is the LPI2C5.
- Make sure the sensor node output is connected to the CSI node input with the correct number of MIPI lanes.
ISI and ISP
To enable the LibCamera path, it is necessary to include the different components that build the path: CSI, ISI and ISP. Refer to i.MX95 camera porting guide in page 5 for details on the full path.
For the ISI path (raw sensor data output), the required components to enable are: dphy, mipi_csi, csi_pixel_formatter and isi. These blocks have cascade interconnection, visit NXP documentation for details.
For the ISP path, the ISP module shall be enabled in addition to the ISI path blocks.
&neoisp0 {
status = "okay";
};
Toradex Yocto build setup
Toradex provides a custom Yocto manifest to build custom versions of the Torizon OS. This wiki uses the Toradex BSP manifest (7.6.1), based on Scarthgap. You can read more about Torizon yocto project in the official Toradex documentation and use RidgeRun's wiki to setup the environment.
New meta layer for Yocto integration
When developing a new driver, and integrating the change using Yocto, it is recommended to do a Camera Software Pack for Yocto. An i.MX camera software pack includes camera drivers, libraries, calibration files, and Yocto recipes. More information about this in NXP software package documentation.
The basic camera SW pack includes:
- conf: directory that holds the layer configuration.
- recipes-kernel: Toradex uses a custom
linux-toradexkernel bsp. When including a new driver, it is necessary to append it to the kernel recipe. - Sensor driver source code and device tree, recommended as an overlay.
- Patch files to include the sensor source code and device tree into the kernel build system.
- Make sure you create this patch files using the correct kernel version according your Toradex BSP. To do so, you can clone a copy of the linux-toradex kernel.
- bbappend kernel file: This append file must overwrite "linux-toradex" recipe, and install the sensor source code, patches, device tree into the linux-toradex kernel source code, to later be compiled.
- README.md and licence: adds documentation to your custom layer.
Porting LibCamera
Toradex uses by default the Scarthgap manifest, which uses an old libCamera version from openEmbedded core layer. It is neccessary to backport the libCamera support into the Yocto build. To port LibCamera into your Toradex Yocto build, it is recommended to create a custom layer, and include the updated libCamera recipes in it. These recipes are from the main NXP libCamera fork and they include main libCamera nxp support and NEO-IPA support.
- Select the LibCamera recipes that best work for you. NXP exposes different LibCamera recipes in their repository. You can choose the branch you wish to work with, and pick the recipes.
- Pick LibCamera recipe, then optionally the IPA uGuzzi recipe.
- Copy the recipes to your custom layer, and rebuild the Yocto image to include their binaries.
- Depending on the LibCamera version you choose, it may be necessary to port the ISP kernel driver from NXP linux repositories. This driver can be included in the custom Yocto layer you create.
RidgeRun already provides instructions on how to port and build LibCamera for your i.MX95.
LibCamera details
LibCamera is a complete framework that bridges the user applications with camera devices. The main goal of the library is to encapsulate the complexity involved in video pipeline configuration for image capturing. The framework is open-source, and it's developed by Linux media community in collaboration with the industry. NXP has adopted libCamera in the architecture of multiple SoC, including i.MX95.
Pipeline handlers are responsible for decoding and processing raw data from the sensor. In the case of raw sensors, they require their output to be passed through the ISP module, and a dedicated handler to control pipeline configuration. NXP already includes a pipeline handler in their LibCamera fork, called NEO-ISP pipeline handler. This handler uses an image-processing algorithm (IPA).
For i.MX95, there are 2 different IPA available:
- NXP IPA: open source algorithm recommended for testing and demonstration.
- uGuzzi IPA: production-level closed source algorithm. UGuzzi support is delivered in a independent library.
The NEO-ISP handler has support for 3A algorithms (auto-exposure, auto-whitebalance and auto-focus), and other algorithms like black level and color corrections and HDR.
The framework has support for a handful of sensors already, but if a new sensor needs to be supported, then the source code must be modified with the necessary information.
NXP Neo-ISP support
The default LibCamera ISP support is using NXP IPA algorithm. To add support for a new sensor, you will have to patch NXP LibCamera source code.
- Clone LibCamera source code and choose the branch you want to use. Newer branches are recommended.
- Apply the changes according i.MX95 sensor porting guide.
- It is required a camera helper class that is specific for the sensor. You can base your development in other existent camera helpers, and adjust the parameters according to the sensor.
- Additionally, you will need a tuning file in the form of a yaml file. This yaml file contains the description of the IPA algorithms to use for the camera, and algorithm parameters.
- Once the changes are ready, you need to include both the camera helper and the yaml file in the build system.
- Rebuild the library to include your new camera helper and configuration file.
Use Neo IPA backend on i.MX95
Configure Neo ISP pipeline to be used by libCamera.
export LIBCAMERA_PIPELINES_MATCH_LIST='nxp/neo,imx8-isi'
By default, LibCamera will use the NXP neo IPA algorithm. You can run a capture pipeline or simply check camera info to confirm that NXP Neo IPA is being used.
LIBCAMERA_LOG_LEVELS=*:DEBUG cam -l
Testing Notes
Make sure the camera is recognized, and the correct pipeline handler (neo-isp) is selected.
export LIBCAMERA_PIPELINES_MATCH_LIST='nxp/neo,imx8-isi' LIBCAMERA_LOG_LEVELS=*:DEBUG cam -l 2>&1 | grep "nxpneo\|nxp/neo"
The expected output should look similar to the following. Make sure you see "nxp/neo matched".
DEBUG Camera camera_manager.cpp:126 Found listed pipeline handler 'nxp/neo' DEBUG NxpNeoPipe neo_utils.cpp:920 Parsing pipeline config file /usr/share/libcamera/pipeline/nxp/neo/config.yaml DEBUG IPAProxy nxpneo_ipa_proxy.cpp:45 initializing nxpneo proxy: loading IPA from /usr/lib/libcamera/ipa_nxp_neo.so INFO NxpNeoIPA neo_ipa.cpp:128 nxpneo: IPANxpNeo NXPNEO_IPA_v0.0.0+5793-lf-6.12.20-2.0.0-1-gaaf99e1d-dirty (2026-07-01T23:01:22UTC) DEBUG NxpNeoIPA neo_ipa.cpp:130 nxpneo: Hardware revision is 0 DEBUG NxpNeoIPA neo_ipa.cpp:131 nxpneo: Sensor entity: sensor123 4-001a DEBUG IPAModuleAlgo module.h:91 nxpneo: Instantiated algorithm 'Awb' DEBUG IPAModuleAlgo module.h:91 nxpneo: Instantiated algorithm 'Agc' DEBUG Camera camera_manager.cpp:161 Pipeline handler "nxp/neo" matched
Check camera capabilities
To check camera capabilities and controls, you have to find the linux subdevice that is registered with the sensor. The id is not fixed, and it will depend on boot order. Use the following command to check the media graph, replace X with the media ID to check:
media-ctl -d /dev/mediaX -p
Use the following commands to print the media format and frame sizes of the attached sensor. Replace "X" with the result of the previous script. The last command will give you the available controls of the camera, like exposure and gain.
v4l2-ctl -d /dev/v4l-subdevX --list-subdev-mbus-codes pad=0 v4l2-ctl -d /dev/v4l-subdevX --list-subdev-framesizes pad=0 v4l2-ctl -d /dev/v4l-subdevX --list-ctrls
LibCamera util example
NeoNXP pipeline handler can automatically configure the pipeline to capture raw bayer frames or debayer frames by changing the "role". Assuming your new sensor can do 1080p, use the following examples to test capture.
cam --camera 1 --stream width=1920,height=1080,role=raw --capture=10 --file=/tmp/frameshd.raw cam --camera 1 --stream width=1920,height=1080 --capture=10 --file=/tmp/frameshddb.raw
Different pixel formats can be specified in the pipe. For more details on pixel formats, use cam --camera 1 --info.
cam --camera 1 --stream width=1920,height=1080,role=video,pixelformat=YUYV --capture=10 --file=/tmp/frames.raw
For testing frame-rate:
cam --camera 1 --stream width=1920,height=1080,pixelformat=YUYV --capture=300 --file=/dev/null Using camera /base/soc/bus@42000000/i2c@426b0000/sensor123@1a as cam0 cam0: Capture 300 frames 181.080041 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 16588800 181.096099 (62.27 fps) cam0-stream0 seq: 000001 bytesused: 16588800 181.112684 (60.30 fps) cam0-stream0 seq: 000002 bytesused: 16588800 181.129851 (58.25 fps) cam0-stream0 seq: 000003 bytesused: 16588800 181.146712 (59.31 fps) cam0-stream0 seq: 000004 bytesused: 16588800
GStreamer examples
To test the libcamerasrc element, you can use the following pipelines. The first one captures to a fakesink. This is used just to test the gst element functionality.
gst-launch-1.0 libcamerasrc src::stream-role=view-finder ! "video/x-raw,width=1920,height=1080,format=YUY2" ! queue ! fakesink
You can capture to a file for later visualization.
gst-launch-1.0 -e libcamerasrc src::stream-role=view-finder ! "video/x-raw,width=1920,height=1080,format=YUY2" ! filesink location=/tmp/frameshd.raw
Since Toradex i.MX95 with Verdin Mallow carrier does not have HDMI support, you can use UDP streaming to visualize the stream in another machine.
gst-launch-1.0 libcamerasrc src::stream-role=view-finder ! "video/x-raw,width=1920,height=1080,format=YUY2,interlace-mode=progressive" ! queue ! v4l2h265enc extra-controls="controls,video_gop_size=60" ! h265parse ! "video/x-h265,stream-format=byte-stream,alignment=au" ! rtph265pay config-interval=-1 pt=96 ! udpsink host=<receiver-ip> port=5000 sync=false
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
