ON Semiconductor AR1335 Linux Driver: Difference between revisions
Jcaballero (talk | contribs) |
Jcaballero (talk | contribs) |
||
Line 417: | Line 417: | ||
--> | --> | ||
|} | |} | ||
{| class="wikitable sortable" | |||
|- | |||
! Nano | |||
|<!-- | |||
--> | |||
{| class="wikitable sortable mw-collapsible mw-collapsed" style="width:100%; margin:0;" | |||
! Feature | |||
! Details | |||
! SDK Support | |||
|- | |||
| 4208x3120@15fps | |||
| 2 Lanes | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| 2104x1560@29fps | |||
| 2 Lanes | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| 512x512@86fps | |||
| 2 Lanes | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| GRBG10 capture | |||
| 4208x3120@15fps only | |||
| L4T 32.5 / Jetpack 4.5, L4T 32.4.3 / Jetpack 4.4 | |||
|- | |||
| GRBG8 capture | |||
| 4208x3120@15fps only | |||
| L4T 32.5 / Jetpack 4.5, L4T 32.4.3 / Jetpack 4.4 | |||
|- | |||
| GREY10 capture | |||
| | |||
| L4T 32.5 / Jetpack 4.5, L4T 32.4.3 / Jetpack 4.4 | |||
|- | |||
| GREY8 capture | |||
| | |||
| L4T 32.5 / Jetpack 4.5, L4T 32.4.3 / Jetpack 4.4 | |||
|- | |||
| V4l2 Media Controller driver | |||
| | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| ISP usage through NvArgusCameraSrc | |||
| Only GRBG10 support | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| Gain control | |||
| | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| Exposure control | |||
| | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| Window control | |||
| | |||
| L4T 32.4.3 / Jetpack 4.4, L4T 32.5 / Jetpack 4.5 | |||
|- | |||
| 2 simultaneous cameras | |||
| | |||
| L4T 32.5 / Jetpack 4.5, L4T 32.4.3 / Jetpack 4.4 | |||
|- | |||
| Reset controls to default value after capture | |||
| | |||
| L4T 32.5 / Jetpack 4.5 | |||
|}<!-- | |||
--> | |||
|- | |||
! Xavier AGX | |||
|<!-- | |||
--> | |||
</center> | </center> | ||
Revision as of 14:38, 18 January 2022
|
|
|
ON Semiconductor AR1335 Features
The AR1335 is a 1/3.2-inch CMOS active-pixel digital image sensor with a pixel array of 4208H x 3120V. The AR1335 digital image sensor, features breakthrough 1.1µm pixel technology that delivers superior low-light image quality through leading sensitivity, quantum efficiency and linear full well. This allows image quality that rivals digital still cameras. With a sensor architecture focused on low power and a high Chief Ray Angle (CRA) for low Z-heights, the AR1335 is ideal for smartphone and other mobile device applications. It incorporates sophisticated on-chip camera functions such as windowing, mirroring, column and row skip modes, and snapshot mode. It is programmable through a simple two-wire serial interface. The AR1335 sensor can generate full resolution image at up to 30 frames per second (fps) and supports advanced video modes including 4K 30fps, 1080P 60fps and 720P 120fps. (Taken from OnSemi webpage).
Learn more about the sensor it the OnSemi webpage!
Supported Platforms
- NVIDIA Jetson TX2
- NVIDIA Jetson Nano
- NVIDIA Jetson Xavier AGX
Features Included in the Driver
Nano |
| |||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
TX2 |
|
Nano |
| |||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Xavier AGX |
Enabling the DriverIn order to use this driver, you have to patch and compile the kernel source using JetPack:
4.2.2_ar1335.patch
Make sure to enable AR1335 driver support: make menuconfig -> Device Drivers -> Multimedia support -> NVIDIA overlay Encoders, decoders, sensors and other helper chips -> <*> AR1335 camera sensor support Driver featuresResolutions supported
V4L2 controls
More features
Using the DriverGStreamer ExamplesCapture and Display
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1' ! autovideosink
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM), width=4208, height=3120, format=NV12, framerate=15/1' ! nvoverlaysink
gst-launch-1.0 v4l2src ! "video/x-bayer, format=grbg, width=4208, height=3120, framerate=15/1" ! capssetter caps="video/x-bayer, format=grbg, width=4224, height=3120" ! bayer2rgb ! videoconvert ! xvimagesink
gst-launch-1.0 v4l2src ! "video/x-raw,width=4208,height=3120,format=GRAY8" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink sync=0
gst-launch-1.0 v4l2src ! "video/x-raw,width=2104,height=1560,format=GRAY8" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink sync=0
gst-launch-1.0 v4l2src ! "video/x-raw,width=512,height=512,format=GRAY8" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink sync=0 Video EncodingCAPS="video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" gst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test.ts The sensor will capture in the 3840x2160@30 mode and the pipeline will encode the video and save it into test.ts file. Dual Capture + Encodinggst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test_0.ts nvarguscamerasrc sensor-id=1 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test_1.ts The sensor will capture from 3 cameras in the 3840x2160@30 mode and the pipeline will encode the videos and save them into test_0.ts and test_1.ts files. Window Control ExamplesROI SelectionThe AR1335 driver exposes x_addr and y_addr controls in order to select the region of interest. These controls can be used for all available resolutions. v4l2-ctl -d /dev/video0 -c x_addr=100 -c y_addr=300 Bin/Scale EnableBy default 1080p mode has an output frame size of about 1920x1080 but covers a full ROI of 3840x2160. However, the binning/scaling process can be disabled using this control to selects a 1920x1080 ROI without changing the frame size. The frame that comes from the camera needs to have a constant size once the streaming begins, so the idea is to start the capture using 1080p mode with the binning/scaling enabled covering the 3840x2160 region and then, select a 1920x1080 ROI in the desired area. This ensures a constant frame size avoiding inconsistent streaming. v4l2-ctl -d /dev/video0 -c bin_scale_enable=0 Analog gainAnalog gain can range from 25 to 2325. v4l2-ctl -c gain=<value> ExposureExposure time is in us, with maximum value being 127595us. v4l2-ctl -c exposure=<value> Digital Gain ExamplesThe driver allows supporting manual white balance handle through red, green, blue digital gain controls. This gain controls only affect the digital contributions to modify the image colors and keep the current analog gain. v4l2-ctl -d /dev/video0 -c red_gain=100 greenr_gain=100 greenb_gain=100 blue_gain=64 Temperature Reading ExampleThe AR1335 temperature sensor works only when the camera sensor is in streaming mode, so the temperature will be set to zero when the sensor is in low-power-mode. The thermal_zone devices registered will be available on /sys/class/thermal/, so you can read temperature as the following: cat /sys/devices/virtual/thermal/thermal_zone8/temp Test Patterns ExampleSupport was added to allow you to enable test pattern mode and select your desired pattern between the following options: 0: Normal operation: no test pattern 1: Solid color 2: 100% color bars 3: Fade−to−gray color bars 4: Walking 1s (10-bit) 5: Walking 1s (8-bit) Try different test patterns selecting a number between 1 and 5: v4l2-ctl -d /dev/video0 -c test_pattern=1 v4l2-ctl -d /dev/video0 -c test_pattern=2 v4l2-ctl -d /dev/video0 -c test_pattern=3 v4l2-ctl -d /dev/video0 -c test_pattern=4 v4l2-ctl -d /dev/video0 -c test_pattern=5 PerformanceARM LoadTegrastats display the following output when capturing with the sensor driver used in the TX2 platform: RAM 1263/7855MB (lfb 1501x4MB) CPU [0%@2035,off,off,0%@2035,0%@2035,0%@2035] RAM 1263/7855MB (lfb 1501x4MB) CPU [23%@960,off,off,17%@960,16%@960,23%@960] RAM 1263/7855MB (lfb 1500x4MB) CPU [17%@345,off,off,17%@345,18%@345,20%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,16%@345,18%@345,15%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,13%@345,15%@345,14%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,15%@345,12%@345,15%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,15%@345,15%@345,16%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,18%@345,18%@345,17%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [16%@345,off,off,15%@345,27%@345,17%@345] RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,18%@345,17%@345,19%@345] FramerateUsing the next pipeline we were able to measure the framerate for single capture with perf element: gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1' ! perf ! fakesink GST-PERF INFO --> Timestamp: 0:07:19.108602798; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:20.141189052; Bps: 782; fps: 30.3 GST-PERF INFO --> Timestamp: 0:07:21.174265435; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:22.207318757; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:23.240543516; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:24.273697886; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:25.306822764; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:26.340117514; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:27.373087284; Bps: 782; fps: 30.3 GST-PERF INFO --> Timestamp: 0:07:28.406069581; Bps: 782; fps: 30.3 GST-PERF INFO --> Timestamp: 0:07:29.439238457; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:30.472398102; Bps: 782; fps: 30.0 GST-PERF INFO --> Timestamp: 0:07:31.472948042; Bps: 808; fps: 30.0 The results show the framerate constant at 30FPS that use nvarguscamerasrc and passing frames through the ISP to convert from Bayer to YUV. Latency measurementThe glass to glass latency measured is about ~150 ms with nvarguscamerasrc.
|