ON Semiconductor AR1335 Linux Driver
 |
|
|
|
ON Semiconductor AR1335 Features
The AR1335 is a 1/3.2-inch CMOS active-pixel digital image sensor with a pixel array of 4208H x 3120V. The AR1335 digital image sensor features breakthrough 1.1µm pixel technology that delivers superior low-light image quality through leading sensitivity, quantum efficiency, and linear full well. This allows image quality that rivals digital still cameras. With a sensor architecture focused on low power and a high Chief Ray Angle (CRA) for low Z-heights, the AR1335 is ideal for smartphones and other mobile device applications. It incorporates sophisticated on-chip camera functions such as windowing, mirroring, column and row skip modes, and snapshot mode. It is programmable through a simple two-wire serial interface. The AR1335 sensor can generate full resolution images at up to 30 frames per second (fps) and supports advanced video modes including 4K 30fps, 1080P 60fps, and 720P 120fps. (Taken from OnSemi webpage).
Learn more about the sensor at the OnSemi webpage!
Supported Platforms
- NVIDIA Jetson TX2
- NVIDIA Jetson Nano
- NVIDIA Jetson Xavier AGX
Features Included in the Driver
| Nano |
| |||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TX2 |
| |||||||||||||||||||||||||||||||||||||||||||||
| Xavier AGX |
|
Enabling the Driver
In order to use this driver, you have to patch and compile the kernel source using JetPack:
Follow the instructions in (Downloading sources) to get the kernel source code.
- Once you have the source code, apply the following three patches in order to add the changes required for the AR1335 camera at kernel and dtb level.
4.2.2_ar1335.patch
- Follow the instructions in (Build Kernel) for building the kernel, and then flash the image.
Make sure to enable AR1335 driver support:
make menuconfig
-> Device Drivers
-> Multimedia support
-> NVIDIA overlay Encoders, decoders, sensors and other helper chips
-> <*> AR1335 camera sensor support
Driver features
Resolutions supported
The following are the modes supported, find examples below:
- 1920x1080@60fps
- 3840x2160@30fps
- 4208x3120@30fps
V4L2 controls
- Gain
- Exposure
- Frame-rate
- Manual White-Balance (digital gain controls)
- Window Control
- Test Patterns
- OTPM Write/Read/Invalidate
More features
- Format: GRGB10
- Number of simultaneous cameras: 2
- Temperature Reading
Using the Driver
GStreamer Examples
Capture and Display
- 3840x2160@30fps GRBG10
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1' ! autovideosink
- 4208x3120@15fps GRBG10
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM), width=4208, height=3120, format=NV12, framerate=15/1' ! nvoverlaysink
- 4208x3120@15fps GRBG8
gst-launch-1.0 v4l2src ! "video/x-bayer, format=grbg, width=4208, height=3120, framerate=15/1" ! capssetter caps="video/x-bayer, format=grbg, width=4224, height=3120" ! bayer2rgb ! videoconvert ! xvimagesink
- 4208x3120@15fps GREY8
gst-launch-1.0 v4l2src ! "video/x-raw,width=4208,height=3120,format=GRAY8" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink sync=0
- 2104x1560@29fps GREY8
gst-launch-1.0 v4l2src ! "video/x-raw,width=2104,height=1560,format=GRAY8" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink sync=0
- 512x512@86fps GREY8
gst-launch-1.0 v4l2src ! "video/x-raw,width=512,height=512,format=GRAY8" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink sync=0
Video Encoding
CAPS="video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" gst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test.ts
The sensor will capture in the 3840x2160@30 mode and the pipeline will encode the video and save it into test.ts file.
Dual Capture + Encoding
gst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test_0.ts nvarguscamerasrc sensor-id=1 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test_1.ts
The sensor will capture from 3 cameras in the 3840x2160@30 mode and the pipeline will encode the videos and save them into test_0.ts and test_1.ts files.
Window Control Examples
ROI Selection
The AR1335 driver exposes x_addr and y_addr controls in order to select the region of interest. These controls can be used for all available resolutions.
v4l2-ctl -d /dev/video0 -c x_addr=100 -c y_addr=300
Bin/Scale Enable
By default 1080p mode has an output frame size of about 1920x1080 but covers a full ROI of 3840x2160. However, the binning/scaling process can be disabled using this control to selects a 1920x1080 ROI without changing the frame size. The frame that comes from the camera needs to have a constant size once the streaming begins, so the idea is to start the capture using 1080p mode with the binning/scaling enabled covering the 3840x2160 region and then, select a 1920x1080 ROI in the desired area. This ensures a constant frame size avoiding inconsistent streaming.
v4l2-ctl -d /dev/video0 -c bin_scale_enable=0
Analog gain
Analog gain can range from 25 to 2325.
v4l2-ctl -c gain=<value>
Exposure
Exposure time is in us, with maximum value being 127595us.
v4l2-ctl -c exposure=<value>
Digital Gain Examples
The driver allows supporting manual white balance handle through red, green, blue digital gain controls. This gain controls only affect the digital contributions to modify the image colors and keep the current analog gain.
v4l2-ctl -d /dev/video0 -c red_gain=100 greenr_gain=100 greenb_gain=100 blue_gain=64
Temperature Reading Example
The AR1335 temperature sensor works only when the camera sensor is in streaming mode, so the temperature will be set to zero when the sensor is in low-power-mode.
The thermal_zone devices registered will be available on /sys/class/thermal/, so you can read temperature as the following:
cat /sys/devices/virtual/thermal/thermal_zone8/temp
Test Patterns Example
Support was added to allow you to enable test pattern mode and select your desired pattern between the following options:
0: Normal operation: no test pattern 1: Solid color 2: 100% color bars 3: Fade−to−gray color bars 4: Walking 1s (10-bit) 5: Walking 1s (8-bit)
Try different test patterns by selecting a number between 1 and 5:
v4l2-ctl -d /dev/video0 -c test_pattern=1 v4l2-ctl -d /dev/video0 -c test_pattern=2 v4l2-ctl -d /dev/video0 -c test_pattern=3 v4l2-ctl -d /dev/video0 -c test_pattern=4 v4l2-ctl -d /dev/video0 -c test_pattern=5
Performance
ARM Load
Tegrastats display the following output when capturing with the sensor driver used in the TX2 platform:
RAM 1263/7855MB (lfb 1501x4MB) CPU [0%@2035,off,off,0%@2035,0%@2035,0%@2035] RAM 1263/7855MB (lfb 1501x4MB) CPU [23%@960,off,off,17%@960,16%@960,23%@960] RAM 1263/7855MB (lfb 1500x4MB) CPU [17%@345,off,off,17%@345,18%@345,20%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,16%@345,18%@345,15%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,13%@345,15%@345,14%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,15%@345,12%@345,15%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,15%@345,15%@345,16%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,18%@345,18%@345,17%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [16%@345,off,off,15%@345,27%@345,17%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,18%@345,17%@345,19%@345]
Framerate
Using the next pipeline we were able to measure the framerate for single capture with perf element:
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1' ! perf ! fakesink
GST-PERF INFO --> Timestamp: 0:07:19.108602798; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:20.141189052; Bps: 782; fps: 30.3 GST-PERF INFO --> Timestamp: 0:07:21.174265435; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:22.207318757; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:23.240543516; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:24.273697886; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:25.306822764; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:26.340117514; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:27.373087284; Bps: 782; fps: 30.3 GST-PERF INFO --> Timestamp: 0:07:28.406069581; Bps: 782; fps: 30.3 GST-PERF INFO --> Timestamp: 0:07:29.439238457; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:30.472398102; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:31.472948042; Bps: 808; fps: 30.0
The results show the framerate constant at 30FPS that use nvarguscamerasrc and passing frames through the ISP to convert from Bayer to YUV.
Latency measurement
The glass to glass latency measured is about ~150 ms with nvarguscamerasrc.
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
