Sony IMX264 Linux Driver
|
|
|
|
Sony IMX264LQR Features
The IMX264 is a CMOS image sensor with a 3.45um pixel and global shutter function for industrial applications. This small-sized 3.45um pixel realizes higher sensitivity and lower noise than the existing 5.86 um pixel products and achieves high picture quality, high resolution, and high-speed imaging without focal plane distortion. In addition, these new CMOS image sensors are equipped with a variety of functions such as a trigger mode that arbitrarily controls the storage time using an external trigger signal, ROI (region of interest) mode. (Taken from Sony's webpage).
Sony IMX264LLR/LQR, IMX265LLR/LQR Sensor Specifications!
Sony IMX264 Linux Driver Support
- NVIDIA Jetson Xavier NX
IMX264 Linux Driver Features
| Xavier NX |
|
|---|
Enabling the IMX264 Linux Driver
In order to use this driver, you have to patch and compile the kernel source using JetPack:
- Follow the instructions in Compiling_Jetson_Xavier_NX_source_code_L4T_32.4.3#Build_NVIDIA_Jetson_Xavier_NX_kernel_source_code Downloading sources to get the kernel source code.
- Once you have the source code, apply the following patch in order to add the changes required for the IMX264 camera at kernel and dtb level.
4.4.1_imx264.patch
- Follow the instructions in Compiling_Jetson_Xavier_NX_source_code_L4T_32.4.3#Compile_Jetson_Xavier_NX_kernel_and_dtb_.28device_tree_blob.29 Build Kernel for building the kernel, and then flash the image.
Make sure to enable IMX264 driver support:
make menuconfig
-> Device Drivers
-> Multimedia support
-> NVIDIA overlay Encoders, decoders, sensors and other helper chips
-> <*> IMX264 camera sensor support
GStreamer Examples: Testing a IMX264 Linux Driver
Capture and Display
- 2656x2088@35fps
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! queue ! nvvidconv ! autovideosink
- 1920x1106@30fps
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=1 ! queue ! nvvidconv ! autovideosink
Capture and Stream
- 2656x2088@35fps
HOST=<Host ip> PORT1=<port1> gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! nvvidconv ! nvv4l2h264enc ! rtph264pay config-interval=10 ! udpsink host=$HOST port=$PORT1
And on the host side:
PORT1=<port1> gst-launch-1.0 udpsrc port=$PORT1 ! 'application/x-rtp, media=(string)video, encoding-name=(string)H264' ! queue ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink
- 1920x1106@30fps
HOST=<Host ip> PORT1=<port1> gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=1 ! nvvidconv ! nvv4l2h264enc ! rtph264pay config-interval=10 ! udpsink host=$HOST port=$PORT1
And on the host side:
PORT1=<port1> gst-launch-1.0 udpsrc port=$PORT1 ! 'application/x-rtp, media=(string)video, encoding-name=(string)H264' ! queue ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink
Analog gain
Analog gain can range from 1 to 480.
v4l2-ctl -c gain=<value>
Exposure
Exposure time is in us, with the minimum value being 52us for all-pixel mode and 43us for Full-HD mode. The maximum value is 36091 for all-pixel mode and 16105 for Full-HD mode. The command to set this value is:
v4l2-ctl -c exposure=<value>
Black level
This value is added to the analog gain. It ranges from 0 to 511. The command to use this control is:
v4l2-ctl -c black_level=<value>
Gain delay
This register determines if the changes are reflected on the current or next frame.
The command to use this is:
v4l2-ctl -c gain_delay=0
Which is the default or
v4l2-ctl -c gain_delay=1
Performance
ARM Load
For a simple pipeline like:
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! queue ! nvvidconv ! autovideosink
Tegrastats displays the following output when capturing with the sensor driver:
RAM 1534/7771MB (lfb 1293x4MB) CPU [31%@1190,27%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [35%@1190,28%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [32%@1190,30%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [31%@1190,31%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [33%@1190,29%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [32%@1190,27%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [32%@1190,28%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [31%@1190,25%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [31%@1190,25%@1190,off,off,off,off] RAM 1534/7771MB (lfb 1293x4MB) CPU [36%@1190,27%@1190,off,off,off,off]
Framerate
Using the next pipeline we were able to measure the framerate for single capture with perf element:
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! perf ! fakesink
INFO: perf: perf0; timestamp: 0:01:08.024467133; bps: 193536.000; mean_bps: 193536.000; fps: 35.685; mean_fps: 35.685 INFO: perf: perf0; timestamp: 0:01:09.032803056; bps: 282240.000; mean_bps: 237888.000; fps: 35.702; mean_fps: 35.694 INFO: perf: perf0; timestamp: 0:01:10.048219125; bps: 290304.000; mean_bps: 255360.000; fps: 35.453; mean_fps: 35.614 INFO: perf: perf0; timestamp: 0:01:11.052939940; bps: 290304.000; mean_bps: 264096.000; fps: 35.831; mean_fps: 35.668 INFO: perf: perf0; timestamp: 0:01:12.066355695; bps: 290304.000; mean_bps: 269337.600; fps: 35.523; mean_fps: 35.639 INFO: perf: perf0; timestamp: 0:01:13.068997375; bps: 282240.000; mean_bps: 271488.000; fps: 35.905; mean_fps: 35.683 INFO: perf: perf0; timestamp: 0:01:14.080078073; bps: 290304.000; mean_bps: 274176.000; fps: 35.605; mean_fps: 35.672
The results show the framerate constant at 35FPS that use nvarguscamerasrc and passing frames through the ISP to convert from Bayer to YUV.
Similarly, for the pipeline:
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=1 ! perf ! fakesink
we obtain:
INFO: perf: perf0; timestamp: 0:02:48.161150666; bps: 169344.000; mean_bps: 169344.000; fps: 30.150; mean_fps: 30.150 INFO: perf: perf0; timestamp: 0:02:49.161479732; bps: 241920.000; mean_bps: 205632.000; fps: 29.990; mean_fps: 30.070 INFO: perf: perf0; timestamp: 0:02:50.193516084; bps: 241920.000; mean_bps: 217728.000; fps: 30.038; mean_fps: 30.059 INFO: perf: perf0; timestamp: 0:02:51.226902182; bps: 241920.000; mean_bps: 223776.000; fps: 29.998; mean_fps: 30.044 INFO: perf: perf0; timestamp: 0:02:52.260178724; bps: 241920.000; mean_bps: 227404.800; fps: 30.002; mean_fps: 30.036 INFO: perf: perf0; timestamp: 0:02:53.260523866; bps: 241920.000; mean_bps: 229824.000; fps: 29.990; mean_fps: 30.028 INFO: perf: perf0; timestamp: 0:02:54.293488648; bps: 241920.000; mean_bps: 231552.000; fps: 30.011; mean_fps: 30.025
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
