Jetson TX2 camera sensors MAX96705 MAX9286 GMSL SERDES Linux Drivers
 |
|
|
|
GMSL SERDES Introduction
The Maxim Gigabit Multimedia Serial Link (GMSL) SERDES technology provides high bandwidth and rich point-to-point interconnections between two endpoints over a single cable, which can be up to 15 meters long. SERDES (serializer/deserializer) technology is widely used for sensors and network communication. SERDES supports multiple protocols over the same link. Automotive industry applications rely heavily on this technology due to its flexibility and performance.
Maxim has developed multiple serializer and deserializer chips to use at both endpoints depending on the application and protocols used. The MAX96705 serializer and the MAX9286 deserializer can work together to provide multimedia solutions. RidgeRun developed a Linux kernel driver to control and use both with the NVIDIA Jetson TX2 and TX1 platforms (Tegra).
MAX96705 serializer and MAX9286 deserializer chips
According to Maxim Website the MAX96705 is a compact serializer with features suited for automotive camera applications and it is function and pin-compatible with the MAX9271 chip. The main features are:
- Works with low-cost 50 ohms coax (100 ohms STP) cables
- Retransmission of control data upon error detection
- Up to 1.74 Gbps serial bitrate
- 12.5 MHz to 87 MHz x 14 Bit + H/V Data
- 36.66 MHz to 116MHz x 12 Bit + H/V Data
- 9,6 kbps to 1 Mbps control channel in UART, I2C, or UART-I2C modes
- Crosspoint switch accepts any input bitmap
- Mode for encoded VSYNC and HSYNC
- 1.7 V to 3.6 I/O supply
At the other endpoint, the MAX9286 is a deserializer able to receive data from up to four GSML serializers (using Virtual Channel IDs) and output data on four CSI-2 lanes. However, in the case of using this deserializer with the Jetson TX1 or TX2, you will be able to connect one camera per deserializer because Jetpack currently doesn't support Virtual Channel IDs on Jetson. The main features of the MAX9286 chip are:
- Works with low-cost 50 ohms coax cables and FAKRA connectors for 100 ohms STP
- Data from image sensors are synchronized to the same pixel
- Automatic internal/external generation of camera sync
- 1 to 4 lane CSI-2 output with 80 Mbps to 1200 Mbps per lane
- 9.6kbps to 1Mbps control channel in UART, mixed UART/I2C, or I2C mode with the clock-stretch capability
- Swappable/selectable serial I/O with swappable polarity
- 2 dedicated GPIO Pins
Using both chips you can build a complete video system with 4 cameras synchronized over 4 serializers and the same deserializer, suitable for surround-view camera systems and Machine vision systems. It is important to mention that Jetson TX1 hardware does not support Virtual Channel IDs (VCID) according to the Technical Reference Manuel (TRM). Moreover, Jetson TX2 hardware supports Virtual Channel IDs BUT the Video Input driver released by NVIDIA does not support VCID at this point so only one camera per deserializer can be used. Figure 1 gives an overview of multiple camera applications at this point (no VCID Support):
Hardware options
Connect Tech offers an Astro Carrier Board ASG012 with 2 embedded MAX9286 deserializer chips which provide the support to connect up to 8 serializers. However, due to limitations previously mentioned related to Virtual Channel IDs on the Jetson Video Input driver provided by NVIDIA only 1 serializer (camera) can be used per deserializer.
- Support: support@connecttech.com
- Sales: sales@connecttech.com
To use the MAX96705 serializer you need to select which sensor are you going to use. Leopard Imaging provides camera solutions with embedded MAX96705 chips:
- OV10635: https://leopardimaging.com/product/li-ov10635-ser/
- OV490: https://leopardimaging.com/product/li-ov10640-490-gmsl/
Sensors - OV10635
The SERDES system can be used with multiple CMOS camera sensors that match the serializer chip requirements. RidgeRun has developed OV10635 and OV10640/OV490 drivers which work with the GMSL system.
OV10635
The Omnivision OV10635 CMOS sensor has the following features:
- Support for image sizes: WXGA (1280x800), HD 720p (1280x720), WVGA (752x480), VGA (640x480), 600x400, CIF (352x288), QVGA (320x240)
- Support for output formats:
- Up to 18-bit combined raw
- Separated 10-bit raw
- 8-/10-bit YUV422
- Output interfaces: 10-bit parallel DVP
- Automatic exposure/gain, auto white balance control, aperture/gamma correction
- Horizontal and vertical windowing capability
- External frame sync capability
- Low power consumption
NVIDIA Jetson TX1/TX2 Driver usage
Testing the driver using YAVTA
Yavta (Yet Another V4L2 Test Application) can be used for testing the driver, according to the input video resolution:
yavta /dev/video0 -c1 -n1 -s1280x720 --enum-formats -fUYVY -Ftest-ov10635.yavta
Also Raw pixels can be used for visualizing the captured frame.
GStreamer examples
720p@30 autovideosink
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=720' ! \ nvvidconv ! autovideosink sync=false
720p@30-h264
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=720' ! \ omxh264enc ! mpegtsmux ! filesink location=test-ov10635-h264.ts -v
720p@30-h265
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=720' ! \ omxh265enc ! mpegtsmux ! filesink location=test-ov10635-h265.ts -v
Dual capture 720p@30-h264
gst-launch-1.0 \ v4l2src num-buffers=300 device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=720' ! \ omxh264enc ! mpegtsmux ! filesink location=test-v4l2-ov10635_0.ts \ v4l2src num-buffers=300 device=/dev/video1 ! 'video/x-raw,format=I420,width=1280,height=720' ! \ omxh264enc ! mpegtsmux ! filesink location=test-v4l2-ov10635_1.ts
Sensors - OV10640 CMOS sensor with OV490 ISP
This is an Omnivision camera solution that involves two chips, the OV10640 sensor and the OV490 ISP, each with special features:
OV10640
- Support for image sizes: 1280x1080, VGA (640x480), 600x400, CIF (352x288), QVGA (320x240) and any cropped size
- Support for output formats:
- 20 bit combined RAW
- 12-bit compressed combined RAW
- Separated 12-bit RAW
- 2x12 bit compressed RAW
- 16-bit log domain combined RAW
- 3x12-bit uncompressed RAW
- OmniHDR-S technology
- Output interfaces:
- 12-bit parallel DVP
- 12-bit MIPI/LDVS CSI-2
- Automatic exposure/gain, auto white balance control, aperture/gamma correction, HDR combination, and tone mapping, Lens correction
- Horizontal and vertical sub-sampling
- External frame sync capability
- Low power consumption
OV490
- Simultaneous output of fully processed YUV for display based applications and RAW only to enable machine vision downstream processing
- Parallel processing of two input streams
- Industry-leading HDR technology, third-generation HDR technology allows capturing of more details in bright and dark environments
- Embedded 32-bit RISC processor for high performance and great flexibility
- Advanced ISP for high-quality image capturing and video streaming
- Advanced auto white balance (AWB)
NVIDIA Jetson TX1/TX2 Driver usage
Testing the driver using YAVTA
Yavta (Yet Another V4L2 Test Application) can be used for testing the driver, according to the input video resolution:
yavta /dev/video0 -c1 -n1 -s1280x1080 --enum-formats -fUYVY -Ftest-ov10490.yavta
Also Raw pixels can be used for visualizing the captured frame.
GStreamer examples
1280x1080@30 autovideosink
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=1080' ! \ nvvidconv ! autovideosink sync=false
1280x1080@30-h264
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=1080' ! \ omxh264enc ! mpegtsmux ! filesink location=test-ov490-h264.ts -v
1280x1080@30-h265
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=1080' ! \ omxh265enc ! mpegtsmux ! filesink location=test-ov490-h265.ts -v
Dual capture 1280x1080@30-h264
gst-launch-1.0 \ v4l2src device=/dev/video0 ! 'video/x-raw,format=I420,width=1280,height=1080' ! \ omxh264enc ! mpegtsmux ! filesink location=test-ov490-h264-0.ts \ v4l2src device=/dev/video1 ! 'video/x-raw,format=I420,width=1280,height=1080' ! \ omxh264enc ! mpegtsmux ! filesink location=test-ov490-h264-1.ts
Sensors - AR0231
The AR0231 is a 1/2.7-inch CMOS digital sensor with a 1928x1208 pixel array.
AR0231
The Omnivision AR0231 CMOS sensor has the following features:
- Key Technologies:
- Automotive Grade Backside Illuminated Pixel
- LED Flicker Mitigation Mode
- Sensor Fault Detection for ASIL−B Compliance
- Up to 4−exposure HDR at 1928 × 1208 and 30 fps or 3−exposure HDR at 1928 × 1208 and 40 fps
- Latest 3.0 mm Back Side Illuminated (BSI) Pixel with ON Semiconductor DR−Pixt Technology
- Data Interfaces: up to 4−lane MIPI CSI−2, Parallel, or up to 4−lane High-Speed Pixel Interface (HiSPi) Serial Interface (SLVS and HiVCM)
- Advanced HDR with Flexible Exposure
- Ratio Control
- LED Flicker Mitigation (LFM) Mode
- Selectable Automatic or User Controlled Black Level Control
- Frame to Frame Switching among up to 4 Contexts to Enable Multi−function Systems
- Spread−spectrum Input Clock Support
- Multi−Camera Synchronization Support
- Multiple CFA Options
NVIDIA Jetson TX1/TX2 Driver usage
Testing the driver using v4l2-ctl
v4l2-ctl is an application from v4l2-utils can be used for testing the driver, according to the input video resolution:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1928,height=1208,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=ar0231.data
Also Raw pixels can be used for visualizing the captured frame.
GStreamer examples
1200p@30 autovideosink
gst-launch-1.0 nvcamerasrc ! "video/x-raw(memory:NVMM),width=(int)1928,height=(int)1208,format=(string)I420,framerate=(fraction)30/1" ! autovideosink
1200p@30-h264
gst-launch-1.0 nvcamerasrc ! "video/x-raw(memory:NVMM),width=(int)1928,height=(int)1208,format=(string)I420,framerate=(fraction)30/1" ! \ omxh264enc ! mpegtsmux ! filesink location=test-ar0231-h264.ts -v
1200p@30-h265
gst-launch-1.0 nvcamerasrc ! "video/x-raw(memory:NVMM),width=(int)1928,height=(int)1208,format=(string)I420,framerate=(fraction)30/1" ! \ omxh265enc ! mpegtsmux ! filesink location=test-ar0231-h265.ts -v
See also
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
