ON Semiconductor AP1302 Linux driver

From RidgeRun Developer Wiki



Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.


Driver List Information
Refer to the RidgeRun Linux Camera Drivers to meet all the list of Drivers available


AP1302 Features

The AP1302 is an image sensor with the following features:

  • Up to 13 Mp (4224x3156) sensor support.
  • 450 Mp/second processing (13Mp at 30/1080p at 120/720p at 240 - subject to sensor limitations).
  • Dual on-chip 32-bit RISC processor cores.
  • Optimized for operation with Aptina's Clarity+™ Pixel technology sensors as well as Bayer pattern CMOS Image Sensors (CIS).
  • Designed to support Aptina sensors that provide video interlaced HDR (iHDR).
  • Color gaining and gamma correction.
  • Frame rate reduction, image resizing and clipping.
  • Auto exposure, auto white balance, auto focus, auto flicker detection and control.
  • Adaptive Tone Mapping, Local Tone Mapping.
  • Scene mode support: scene selection and automatic detection.
  • Preferred Color reproduction.
  • Motion-assisted anti-shake with ISO control, face detection and smile detection.
  • Video (Movie) stabilization assist for video resolutions up to 1080p.
  • Batch-mode capture, Best-picture assist.
  • Smooth digital zoom up to 16X.
  • Exposure bracketing for improved dynamic range.
  • OSD, special effects.
  • Test pattern generator.
  • Programmable JPEG encoder with EXIF header support.
  • Scalado SpeedTagsTM Encode support.
  • Two-wire serial interface (I2C) for sensor and peripheral control and register access supporting Standard (100kbps), Fast mode (400kbps), FM+ (1Mbps) and HS (3.4 Mbps).
  • Dual sensor support (second camera or 3D bridge application).
  • 6 MIPI data lanes shared between two sensor interfaces (4+2/3+3/4+1/3+2).
  • SPI External EEPROM/FLASH (4 M-bit) support for code and/or configuration data storage. Additional four-wire serial interface (SPI) for register access supporting up to 25 Mbps.
  • 12 GPIOs (shared functionality with SPI master and slave, second I2C master).
  • Xenon, LED Flash, pulsed LED Flash Support.
  • Fail-safe IO, programmable slew-rate control.

RidgeRun has developed a driver for the Jetson TX2 with the following support:

  • L4T 32.2.0 and Jetpack 4.2
  • V4L2 Media controller driver
  • Supported Formats:
    • Raw10
    • UYVY
  • Supported Resolutions:
    • 640x360@30
    • 640x400@30
    • 1280x720@30
    • 1920x1080@30
    • 3840x2160@25
    • 3968x2160@25
    • 4096x2160@20
    • 3968x3120@15
    • 4208x3120@15

Enabling the driver

In order to use this driver, you have to patch and compile the kernel source using JetPack:

  • Once you have the source code, apply the following patche in order to add the changes required for the AP1302 camera at kernel and dtb level.
4.2_ap1302.patch
  • Follow the instructions in (Build Kernel) for building the kernel, and then flash the image.

Make sure to enable AP1302 driver support: 

make menuconfig

-> Device Drivers                                                                                                                        
  -> Multimedia support                                                                                           
    -> NVIDIA overlay Encoders, decoders, sensors and other helper chips
       -> <*> AP1302 camera sensor support

Using the driver

GStreamer examples

Capture

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=(int)4208, height=(int)3120, format=(string)UYVY' ! autovideosink

The sensor will capture in the 4208x3120@15 mode 

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Video Encoding

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=(int)4208, height=(int)3120, format=(string)UYVY' ! nvvidconv ! omxh264enc ! mpegtsmux ! filesink location=test.ts

The sensor will capture in the 4208x3120@15 mode and the pipeline will encode the video and save it into test.ts file.

Performance

ARM Load

Tegrastats display the following output when capturing with the sensor: 

RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [27%@2035,off,off,17%@2035,12%@2035,9%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [31%@2035,off,off,17%@2035,13%@2035,4%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [27%@2035,off,off,18%@2035,9%@2035,13%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [51%@2035,off,off,4%@2035,9%@2035,3%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [51%@2035,off,off,4%@2035,8%@2035,0%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [44%@2035,off,off,0%@2035,22%@2035,0%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [46%@2035,off,off,8%@2035,13%@2035,0%@2035]

Framerate

Using the next pipeline we were able to measure the framerate for single capture with the perf element: 

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=(int)4208, height=(int)3120, format=(string)UYVY' ! perf ! fakesink

INFO: perf: perf0; timestamp: 0:20:41.612503008; bps: 0,000; mean_bps: 0,000; fps: 0,000; mean_fps: 0,000
INFO: perf: perf0; timestamp: 0:20:42.641163584; bps: 3279792536,737; mean_bps: 3279792536,737; fps: 15,554; mean_fps: 15,554
INFO: perf: perf0; timestamp: 0:20:43.684277280; bps: 3234348559,450; mean_bps: 3257070548,093; fps: 15,339; mean_fps: 15,446
INFO: perf: perf0; timestamp: 0:20:44.728300096; bps: 3231532135,405; mean_bps: 3248557743,864; fps: 15,325; mean_fps: 15,406
INFO: perf: perf0; timestamp: 0:20:45.773840800; bps: 3226840683,574; mean_bps: 3243128478,791; fps: 15,303; mean_fps: 15,380
INFO: perf: perf0; timestamp: 0:20:46.816163744; bps: 3236802278,431; mean_bps: 3241863238,719; fps: 15,350; mean_fps: 15,374
INFO: perf: perf0; timestamp: 0:20:47.862452032; bps: 3224535071,925; mean_bps: 3238975210,920; fps: 15,292; mean_fps: 15,361
INFO: perf: perf0; timestamp: 0:20:48.903999360; bps: 3239212649,586; mean_bps: 3239009130,730; fps: 15,362; mean_fps: 15,361
INFO: perf: perf0; timestamp: 0:20:49.950199680; bps: 3224806201,550; mean_bps: 3237233764,582; fps: 15,293; mean_fps: 15,352
INFO: perf: perf0; timestamp: 0:20:50.991857952; bps: 3238867650,446; mean_bps: 3237415307,456; fps: 15,360; mean_fps: 15,353
INFO: perf: perf0; timestamp: 0:20:52.035787040; bps: 3231822274,886; mean_bps: 3236856004,199; fps: 15,327; mean_fps: 15,351
INFO: perf: perf0; timestamp: 0:20:53.082210528; bps: 3224118455,567; mean_bps: 3235698045,232; fps: 15,290; mean_fps: 15,345
INFO: perf: perf0; timestamp: 0:20:54.126004192; bps: 3232241578,351; mean_bps: 3235410006,326; fps: 15,329; mean_fps: 15,344
INFO: perf: perf0; timestamp: 0:20:55.167571168; bps: 3239151545,450; mean_bps: 3235697817,028; fps: 15,361; mean_fps: 15,345
INFO: perf: perf0; timestamp: 0:20:56.211503776; bps: 3231811377,617; mean_bps: 3235420214,212; fps: 15,327; mean_fps: 15,344
INFO: perf: perf0; timestamp: 0:20:57.255406240; bps: 3231904700,246; mean_bps: 3235185846,615; fps: 15,327; mean_fps: 15,343
INFO: perf: perf0; timestamp: 0:20:58.301784736; bps: 3224257085,650; mean_bps: 3234502799,054; fps: 15,291; mean_fps: 15,339
INFO: perf: perf0; timestamp: 0:20:59.343773824; bps: 3237839358,256; mean_bps: 3234699067,243; fps: 15,355; mean_fps: 15,340

The results show the framerate constant at 15FPS that uses v4l2src.


For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.


Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.

COMPANY

SUPPORT

ARTIFICIAL INTELLIGENCE

© 2024 RidgeRun. All rights reserved.


Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=ON_Semiconductor_AP1302_Linux_driver&oldid=59129"