Mira130 Linux Driver
|
|
|
AMS MIRA130 Features
The Mira130 is a global shutter CMOS and monochrome sensor with an effective pixel array output of 1080 H x 1280 V. This sensor supports NIR enhancement of the QE, and operations such as high dynamic range (HDR) mode, external triggering, windowing, horizontal or vertical mirroring. This sensor can perform a framerate of 120 fps with 10-bit data at a resolution of 1080 H x 1280 V as a maximum. This chip operates with analog 2.5 V, digital 1.8 V, and interface 1.8 V. High sensitivity, programmable registers through I2C, low power consumption, build-in temperature sensor are features that this sensor provides. (Applications: 3D structured light, 3D Active Stereo systems, Machine vision)
Supported Platforms
- NVIDIA Jetson Nano Development Kit B01
Features Included in the Driver
Nano |
|
---|
RidgeRun has developed a driver for the Jetson Nano platform with the following support:
- V4l2 Media controller driver
- Capture with GStreamer v4l2src and v4l2-ctl
Enabling the driver
To use this driver, you have to patch and compile the kernel source.
Using Jetpack
Follow these instructions:
1. Download the toolchain following the instructions from:
Download and install the Toolchain
2. Follow the instructions to download and install the NVIDIA SDK Manager from:
Nvidia SDK Manager
- Then choose the platform (Jetson Nano) and version of JetPack (4.6).
-The Nvidia SDK manager are going to install in a directory similar to:
$HOME/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/
3. Get the L4T Nano sources from:
cd $HOME/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/ ./source_sync.sh -t tegra-l4t-r32.6.1
4. Apply the contents provided in 4.6_evm_mira130_v0.1.tar in the sources directory:
- First untar the provided tarball:
tar -xvf 4.3_ovm6211-v0.1.0.tar
You can then apply the patch:
quilt push -a
6. To compile the code follow the steps:
export DEVDIR=$HOME/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra export PATCHESPATH=$HOME/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/sources/patches/ cd $DEVDIR
# Create the directory to store the compiled image and dtb mkdir -p $DEVDIR/images/dtb
export TEGRA_KERNEL_OUT=$DEVDIR/images export ARCH=arm64 export KERNEL_DIR=$DEVDIR/sources/kernel/kernel-4.9 export CROSS_COMPILE=$HOME/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- export LOCALVERSION=-tegra
cd $KERNEL_DIR
make mrproper
- Make sure to enable MIRA130 driver support:
make O=$TEGRA_KERNEL_OUT tegra_defconfig make O=$TEGRA_KERNEL_OUT menuconfig
- In the terminal menu that appears, select:
Note: By default, the driver is selected with an asterisk. For that reason, if you go back by hitting the double Esc key the message: Do you want to save your new configuration? will not appear.
Device Drivers ---> <*> Multimedia support ---> NVIDIA overlay Encoders, decoders, sensors and other helper chips ---> <*> MIRA130 camera sensor support
If the driver is not selected, press the Y key in order to select the MIRA130 option. Go back by hitting the double Esc key until you get the message: Do you want to save your new configuration?, select Yes and press Enter'
- Compile the kernel:
make O=$TEGRA_KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j4 zImage
- Compile the device tree:
make O=$TEGRA_KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j4 dtbs
8. Flash the Jetson Nano:
Make sure the Jetson Nano is in recovery mode.
- Copy the compiled image to the kernel directory.
cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image $TEGRA_KERNEL_OUT/arch/arm64/boot/zImage $DEVDIR/kernel/
- Copy the compiled device tree to the kernel directory.
cp -r $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/* $DEVDIR/kernel/dtb/
- Flash the memory following the next guide:
cd $DEVDIR sudo ./flash.sh jetson-nano-qspi-sd mmcblk0p1
- Reboot the board after the flashing is completed.
Apply the v4l2src patch
In order to capture with v4l2src, a patch needs to be applied to gstreamer in order for v4l2src to support a Y10 format output.
1. Please extract the contents provided in extra_gstreamer_flashing_patches.tar
in sources/patches
directory:
cd $PATCHESPATH
tar -xvf extra_gstreamer_flashing_patches.tar
Apply the v4l2src patch to the Jetson Nano Devkit board
1. Transfer the patch to the board:
cd $PATCHESPATH
scp add-Y10-support-1.14.5.patch <nvidia-nano-user>@<nvidia-nano-ip>:/home/<nvidia-nano-username>
2. In the board, download the necessary gstreamer dependencies:
sudo apt update
sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
3. Download gst-plugins-good:
mkdir gstreamer-1.14.5; cd gstreamer-1.14.5
wget https://gstreamer.freedesktop.org/src/gst-plugins-good/gst-plugins-good-1.14.5.tar.xz
tar -xvf gst-plugins-good-1.14.5.tar.xz
4. Apply the patch:
cd gst-plugins-good-1.14.5/sys/v4l2/
patch -i $HOME/add-Y10-support-1.14.5.patch
Compile and install
1. Compile:
cd ~/gstreamer-1.14.5/gst-plugins-good-1.14.5
./configure --prefix=/usr --libdir=/usr/lib/aarch64-linux-gnu/
make
DESTDIR=$(pwd)/install make install
2. Install the library:
sudo cp install/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
Using the Driver
Capture with v4l2-ctl
- Install v4l utils:
sudo apt install v4l-utils
- Test the capture framerate:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1080,height=1280,pixelformat=Y10 --set-ctrl bypass_mode=0 --stream-mmap
The output should look like the following:
ams@ams-desktop:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1080,height=1280,pixelformat=Y10 --set-ctrl bypass_mode=0 --stream-mmap <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.00 fps <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.00 fps <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.00 fps <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.00 fps <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.00 fps <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 120.00 fps
- Capture a single frame:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1080,height=1280,pixelformat=Y10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test_frame_120fps.raw
The MIRA130 supports a resolution of 1080x1280 but the platform used (Jetson Nano) defined padding of 8 pixels to the image in order to align and optimize the capture process. The post-process applied is to open the image with a 1088x1280 resolution using Vooya o Rawpixels.
Please consider the following settings to be able to view it correctly:
- rawpixels:
- width: 1088
- height: 1280
- Predefined format: Grayscale 8bit
- Pixel format: Grayscale
- bpp1: 16
- Little Endian box checked
Note: The Y10 format comes in 2 bytes with a padding of zeros for the remaining bits. Since rawpixels requires the bit depth specification for Y10 to be 16bpps, the image looks darker.
- vooya:
- width: 1088
- height: 1280
- Frames/Second: 120
- Color Space: Single Channel
- Data Container: Single Integer
- Bit Depth (Value): 14bit
Note1: Bit depth needs to be 14 due to how the Jetson boards pack pixels. The image looks darker.
Note2: To see the frame the bit depth needs to be 9 or 10.
Capture with Gstreamer
Important: Since Y10 is not supported by v4l2src, GRAY16_LE is used for buffer dequeue, this causes GStreamer to take the full 2 bytes instead of just the 10 bits, which results in a darker image due to a 10bit image being interpreted as a 16bit image.
Important: To achieve the maximum performance at 120 fps on the Jetson Nano Devkit board, you need to run the following command on the terminal everytime the board is power on:
sudo jetson_clocks
Performance statistics
gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,width=1080,height=1280,framerate=120/1,format=GRAY16_LE" ! queue ! fakesink
RAM 1233/3963MB (lfb 400x4MB) CPU [0%@1479,0%@1479,0%@1479,50%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [1%@1479,0%@1479,0%@1479,52%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [3%@1479,0%@1479,0%@1479,52%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [8%@1479,1%@1479,0%@1479,51%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [2%@1479,0%@1479,0%@1479,52%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [1%@1479,0%@1479,0%@1479,51%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [1%@1479,1%@1479,0%@1479,51%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [2%@1479,2%@1479,16%@1479,37%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [0%@1479,0%@1479,0%@1479,51%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [0%@1479,0%@1479,45%@1479,7%@1479] RAM 1233/3963MB (lfb 400x4MB) CPU [1%@1479,1%@1479,52%@1479,0%@1479]
Framerate
Using the next pipeline we were able to measure the framerate for single capture with perf element:
gst-launch-1.0 v4l2src device=/dev/video0 ! perf ! "video/x-raw,width=1080,height=1280,framerate=120/1,format=GRAY16_LE" ! fakesink
perf: perf0; timestamp: 0:56:38.972435551; bps: 0,000; mean_bps: 0,000; fps: 0,000; mean_fps: 0,000 INFO: perf: perf0; timestamp: 0:56:39.979731288; bps: 2540175360,000; mean_bps: 0,000; fps: 120,124; mean_fps: 120,124 INFO: perf: perf0; timestamp: 0:56:40.979764633; bps: 2673868800,000; mean_bps: 2673868800,000; fps: 119,996; mean_fps: 120,060 INFO: perf: perf0; timestamp: 0:56:41.979774896; bps: 2673868800,000; mean_bps: 2673868800,000; fps: 119,999; mean_fps: 120,039 INFO: perf: perf0; timestamp: 0:56:42.979792024; bps: 2673868800,000; mean_bps: 2673868800,000; fps: 119,998; mean_fps: 120,029 INFO: perf: perf0; timestamp: 0:56:43.979810601; bps: 2673868800,000; mean_bps: 2673868800,000; fps: 119,998; mean_fps: 120,023 INFO: perf: perf0; timestamp: 0:56:44.979836408; bps: 2673868800,000; mean_bps: 2673868800,000; fps: 119,997; mean_fps: 120,018 INFO: perf: perf0; timestamp: 0:56:45.979851268; bps: 2696151040,000; mean_bps: 2677582506,667; fps: 119,998; mean_fps: 120,016 INFO: perf: perf0; timestamp: 0:56:46.979881952; bps: 2673868800,000; mean_bps: 2677051977,143; fps: 119,996; mean_fps: 120,013 INFO: perf: perf0; timestamp: 0:56:47.979909813; bps: 2673868800,000; mean_bps: 2676654080,000; fps: 119,997; mean_fps: 120,011 INFO: perf: perf0; timestamp: 0:56:48.979927144; bps: 2673868800,000; mean_bps: 2676344604,444; fps: 119,998; mean_fps: 120,010 INFO: perf: perf0; timestamp: 0:56:49.979933060; bps: 2673868800,000; mean_bps: 2676097024,000; fps: 119,999; mean_fps: 120,009 INFO: perf: perf0; timestamp: 0:56:50.979961414; bps: 2673868800,000; mean_bps: 2675894458,182; fps: 119,997; mean_fps: 120,008 INFO: perf: perf0; timestamp: 0:56:51.979984914; bps: 2673868800,000; mean_bps: 2675725653,333; fps: 119,997; mean_fps: 120,007 INFO: perf: perf0; timestamp: 0:56:52.980041219; bps: 2673868800,000; mean_bps: 2675582818,462; fps: 119,993; mean_fps: 120,006 INFO: perf: perf0; timestamp: 0:56:53.988375122; bps: 2696151040,000; mean_bps: 2677051977,143; fps: 120,000; mean_fps: 120,006 INFO: perf: perf0; timestamp: 0:56:54.996700006; bps: 2673868800,000; mean_bps: 2676839765,333; fps: 120,001; mean_fps: 120,005 INFO: perf: perf0; timestamp: 0:56:55.996715395; bps: 2673868800,000; mean_bps: 2676654080,000; fps: 119,998; mean_fps: 120,005 INFO: perf: perf0; timestamp: 0:56:56.996737859; bps: 2673868800,000; mean_bps: 2676490240,000; fps: 119,997; mean_fps: 120,005 INFO: perf: perf0; timestamp: 0:56:57.996773699; bps: 2673868800,000; mean_bps: 2676344604,444; fps: 119,996; mean_fps: 120,004 INFO: perf: perf0; timestamp: 0:56:58.996792290; bps: 2673868800,000; mean_bps: 2676214298,947; fps: 119,998; mean_fps: 120,004 INFO: perf: perf0; timestamp: 0:56:59.996808788; bps: 2673868800,000; mean_bps: 2676097024,000; fps: 119,998; mean_fps: 120,004 INFO: perf: perf0; timestamp: 0:57:00.996826163; bps: 2673868800,000; mean_bps: 2675990918,095; fps: 119,998; mean_fps: 120,003 INFO: perf: perf0; timestamp: 0:57:01.996847486; bps: 2696151040,000; mean_bps: 2676907287,273; fps: 119,997; mean_fps: 120,003
RidgeRun Resources | |||||
Contact Us
|