OmniVision OV5647 Linux driver
 |
|
|
|
Keywords: OV5647 Jetson TX1, GStreamer, Raspberry PI, NVIDIA, RidgeRun, V4L2 Driver, OmniVision
OmniVision OV5647 Linux driver overview video
OV5647 features
The OmniVision OV5647 is a CMOS image sensor with the following features:
- Automatic image control functions:
- Automatic exposure control (AEC)
- Automatic white balance (AWB)
- Automatic band filter (ABF)
- Automatic 50/60 Hz luminance detection
- Automatic black level calibration (ABLC)
- Image quality controls: lens correction, defective pixel canceling
- CSI2 serial data output (MIPI interface 2 lanes)
- Support for output formats: 8-/10-bit raw RGB data
- Maximum image transfer rate:
- QSXGA "Mpixel" (2592 x 1944): 15 fps
- 1080p: 30 fps
- 960p: 45 fps
- 720p: 60 fps
- VGA (640 x 480): 90 fps
- QVGA (320 x 240): 120 fps
RidgeRun has developed a driver for the Jetson TX1 platform with the following support:
- V4l2 Media controller driver
- Tested resolution 1080p @ 30 fps. (More resolutions support are in the developing process).
- Output format: RAW10 Bayer BGGR pattern.
- Tested with J100 and J20 Auvidea boards
- Capture with v4l2src and nvcamerasrc
- Support for sextuple video capture. (Tested using the 6 available ports on Auvidea J20 expansion board)
Currently available for:
- OV5647 - TX1 - Jetpack 3.1 - J100/J20
- OV5647 - TX2 - Jetpack 3.3 - J100/J20/J90
- OV5647 - TX1 - Jetpack 3.3 - J100 (work in progress)
Enabling the driver
In order to use this driver, you have to patch and compile the kernel source.
Using Jetpack
- Follow the instructions in Compiling Jetson TX1/TX2 source code (Downloading the code) to get the kernel source code.
- If you didn't have download the toolchain yet, follow the instructions in [[Compiling_Jetson_TX1/TX2_source_code#Toolchain Compiling Jetson TX1/TX2 source code (Toolchain)]
- Once you have the source code, apply the patches for fix kernel error during compilation and add the support for the OV5647:
cd $DEVDIR/64_TX1/Linux_for_Tegra_64_tx1/sources/ PATCHES=<path to the tarball> tar -xzf $PATCHES/ov5647-driver-for-tegra-x1-patches.tar.gz mv ov5647-driver-for-tegra-x1-patches patches quilt push -a
Please change the path in PATCHES for the one where you downloaded the tarball with the OV5647 patches.
- Follow the instructions in Compiling Jetson TX1/TX2 source_code (Build Kernel) for building the kernel, and then flash the image. Omit steps 3 and 4 because you already applied the patches.
Make sure to enable OV5647 and J20 driver support in step 5:
-> Device Drivers
-> Multimedia support
-> Encoders, decoders, sensors and other helper chips
-> <*> Auvidea J20 Expansion board
-> <*> Omnivision OV5647 camera sensor support
- To flash the built image into the board follow the instructions in Compiling Jetson TX1/TX2 source_code (Flashing_the_board)
Using the driver (GStreamer examples)
The GStreamer version distributed with Jetpack doesn't support Bayer RAW10 only RAW8 so GStreamer needs to be patched in order to capture using v4l2src. Follow the steps in the following wiki page to add the support for RAW10:
Compile_gstreamer_on_Jetson_TX1_and_TX2
 | Important Note: When you are accessing the board through serial or ssh and you want to run a pipeline to display with autovideosink, nveglglessink, xvimagesink or any other video sink, you have to run your pipeline with DISPLAY=:0 at the beginning of the description: |
DISPLAY=:0 gst-launch-1.0 ...
| Note: These tests were done using the J20 board from Auvidea. |
Getting_started_guide_for_Auvidea_J20_board
Demo Video Pipeline Example
By following this link Overview Demo Video you can see the results of executing the below pipeline and get some CPU/GPU load percentage and framerate measurements.
The below pipeline executes the following actions:
- Sextuple RGB video capture at 1080p@30fps
- Downscale to 320x240 and color-space format conversion to YUV I420
- Mix of the 6 streams into one window (Mosaic)
- Display
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, \ height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, \ height=(int)240, format=(string)I420, framerate=(fraction)30/1' ! perf ! mixer.sink_1 nvcamerasrc sensor-id=1 \ fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \ framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, height=(int)240, format=(string)I420, \ framerate=(fraction)30/1' ! mixer.sink_2 nvcamerasrc sensor-id=2 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \ width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, \ width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)30/1' ! mixer.sink_3 nvcamerasrc \ sensor-id=3 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \ framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, height=(int)240, format=(string)I420, \ framerate=(fraction)30/1' ! mixer.sink_4 nvcamerasrc sensor-id=4 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \ width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, \ width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)30/1' ! mixer.sink_5 nvcamerasrc \ sensor-id=5 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \ framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, height=(int)240, format=(string)I420, \ framerate=(fraction)30/1' ! mixer.sink_6 videotestsrc pattern="black" ! video/x-raw,width=1,height=1 ! videomixer \ name=mixer sink_0::xpos=0 sink_0::ypos=0 sink_0::alpha=0 sink_1::xpos=320 sink_1::ypos=0 sink_2::xpos=320 \ sink_2::ypos=240 sink_3::xpos=0 sink_3::ypos=0 sink_4::xpos=0 sink_4::ypos=480 sink_5::xpos=0 sink_5::ypos=240 \ sink_6::xpos=320 sink_6::ypos=480 ! queue ! ximagesink sync=false -v
This is a snapshot of the Jetson TX1 display while running the above pipeline:
Performance statistics
- Tegrastats in normal operation:
RAM 827/3995MB (lfb 660x4MB) cpu [4%,3%,2%,6%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [6%,2%,3%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [1%,1%,2%,10%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [4%,4%,5%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [3%,1%,4%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [6%,3%,2%,6%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [3%,1%,2%,10%]@102 GR3D 0%@76 EDP limit 0 RAM 828/3995MB (lfb 660x4MB) cpu [9%,2%,2%,2%]@102 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [9%,3%,2%,3%]@403 GR3D 0%@76 EDP limit 0 RAM 827/3995MB (lfb 660x4MB) cpu [0%,0%,3%,0%]@102 GR3D 0%@76 EDP limit 0
- Tegrastats with the above pipeline running
RAM 1821/3995MB (lfb 404x4MB) cpu [57%,56%,60%,48%]@1734 GR3D 44%@76 EDP limit 0 RAM 1819/3995MB (lfb 404x4MB) cpu [59%,58%,54%,55%]@1734 GR3D 20%@76 EDP limit 0 RAM 1821/3995MB (lfb 404x4MB) cpu [53%,66%,52%,56%]@1734 GR3D 28%@76 EDP limit 0 RAM 1822/3995MB (lfb 404x4MB) cpu [57%,66%,53%,54%]@1734 GR3D 42%@76 EDP limit 0 RAM 1820/3995MB (lfb 404x4MB) cpu [50%,61%,52%,53%]@1734 GR3D 46%@76 EDP limit 0 RAM 1822/3995MB (lfb 404x4MB) cpu [55%,63%,52%,59%]@1734 GR3D 29%@76 EDP limit 0 RAM 1822/3995MB (lfb 404x4MB) cpu [56%,62%,56%,51%]@1734 GR3D 42%@76 EDP limit 0 RAM 1820/3995MB (lfb 404x4MB) cpu [55%,64%,52%,56%]@1734 GR3D 20%@76 EDP limit 0 RAM 1822/3995MB (lfb 404x4MB) cpu [57%,62%,50%,55%]@1734 GR3D 34%@76 EDP limit 0 RAM 1822/3995MB (lfb 404x4MB) cpu [59%,66%,49%,57%]@1734 GR3D 34%@76 EDP limit 0
Snapshots
In order to check the snapshot, you can use the following tool:
https://github.com/jdthomas/bayer2rgb
So, run the following commands to download the tool and compile it:
git clone git@github.com:jdthomas/bayer2rgb.git cd bayer2rgb make cp bayer2rgb /usr/bin/
Bayer2rgb will convert naked (no header) Bayer grid data into RGB data. There are several choices of interpolation (though they all look essentially the same to my eye). It can output tiff files and can integrate with ImageMagick to output other formats.
- 1920x1080
gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=bggr, width=1920, height=1080" \ ! multifilesink location=test%d_1920x1080.bayer
Check the snapshot with:
./bayer2rgb --input=test#_1920x1080.bayer --output=data.tiff --width=1920 --height=1080 --bpp=16 --first=BGGR \ --method=BILINEAR --tiff
Use image_magik to convert the tiff to png:
convert data.tiff data.png
| Important Note 1: In general the first buffer contains very low light because the AWB algorithm of the sensor is calibrating, so we recommend using multifilesink to test debayer with a buffer above from number one. To obtain better image colors and bright quality, due to automatic sensor image calibration, we recommend testing debayer with a frame above number 10, to give time to the sensor to adjust the best image calibration parameters. |
| Important Note 2: The debayered image obtained as the output when using the "bayer2rgb" tool presents some kind of light saturation, viewed as multiple color pixels sections. This is a problem of the tool used to do the debayer process but led the users to verify that the driver and camera sensor is working fine. |
Snapshots with nvcamerasrc
The following pipeline will create a file for each captured frame. You can visualize the file on the following web page: http://rawpixels.net/
gst-launch-1.0 -v nvcamerasrc sensor-id=1 fpsRange="30 30" num-buffers=100 ! 'video/x-raw(memory:NVMM), width=(int)1920, \ height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)1920, \ height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! multifilesink location=test_%d.yuv
Single Capture
V4l2src
You can use the raw2rgbpnm tool to check all the buffers:
https://github.com/martinezjavier/raw2rgbpnm
So, run the following commands to download the tool and compile it:
git clone git clone git@github.com:martinezjavier/raw2rgbpnm.git cd raw2rgbpnm
Open the file raw2rgbpnm.c and change line 489 with:
int c = getopt(argc, argv, "a:b:f:ghs:wn");
This is to enable the option to extract multiple frames from a file. Now, you can build the application:
make
| Important Note: This tool converts from GRBG10 to pnm. We capture BGGR in the OV5647, so you will see that the colors at the output of the image are wrong. |
In order to capture 10 buffers and save them in a file, you can run the following pipelines:
- 1920x1080
gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=bggr, width=1920, height=1080" \ ! filesink location=test_1920x1080.bayer
Check the buffers with:
./raw2rgbpnm -f SGRBG10 -s 1920x1080 -b 5.0 -n test_1920x1080.bayer output_1920x1080
Nvcamerasrc
- 1920x1080
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, \ height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink -e
This is an image captured with the above pipeline:
Sextuple Capture
The following image consists of a Jetson TX1 with six ov5647 cameras plugged in the Auvidea J20 expansion board and doing video capture at the same time:
Sextuple Capture Connection Diagram:
Using the following pipelines we can test the performance of the Jetson TX1 when doing sextuple capture:
V4l2src
- Pipeline for sextuple video capture using v4l2src, at 1920x1080 @30fps:
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \ v4l2src device=/dev/video1 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \ v4l2src device=/dev/video2 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \ v4l2src device=/dev/video3 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \ v4l2src device=/dev/video4 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \ v4l2src device=/dev/video5 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink
Performance statistics
- Tegrastats in normal operation:
RAM 1579/3994MB (lfb 378x4MB) cpu [3%,5%,2%,7%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [3%,4%,5%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [2%,1%,8%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [4%,2%,8%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [2%,3%,7%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [3%,3%,6%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [5%,1%,5%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [1%,2%,7%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1579/3994MB (lfb 378x4MB) cpu [8%,1%,1%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1578/3994MB (lfb 378x4MB) cpu [8%,2%,2%,6%]@102 GR3D 0%@76 EDP limit 0
- Tegrastats with the above pipeline running:
RAM 1677/3994MB (lfb 370x4MB) cpu [20%,3%,5%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [18%,5%,6%,2%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [18%,3%,4%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [18%,7%,3%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [11%,9%,7%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [17%,9%,4%,2%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [17%,4%,8%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [15%,6%,2%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [21%,6%,4%,7%]@102 GR3D 0%@76 EDP limit 0 RAM 1677/3994MB (lfb 370x4MB) cpu [19%,3%,5%,4%]@102 GR3D 0%@76 EDP limit 0
Nvcamerasrc
- Pipeline for sextuple video capture using nvcamerasrc, at 1920x1080 @30fps:
gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=1 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \ width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=2 \ fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' \ ! fakesink nvcamerasrc sensor-id=3 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=4 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \ width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=5 \ fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' \ ! fakesink
Performance statistics
- Tegrastats in normal operation:
RAM 1490/3994MB (lfb 412x4MB) cpu [5%,3%,5%,2%]@102 GR3D 0%@76 EDP limit 0 RAM 1490/3994MB (lfb 412x4MB) cpu [5%,1%,5%,3%]@307 GR3D 0%@76 EDP limit 0 RAM 1490/3994MB (lfb 412x4MB) cpu [5%,2%,4%,0%]@102 GR3D 0%@76 EDP limit 0 RAM 1490/3994MB (lfb 412x4MB) cpu [8%,1%,6%,2%]@102 GR3D 0%@76 EDP limit 0 RAM 1490/3994MB (lfb 412x4MB) cpu [7%,5%,6%,9%]@102 GR3D 0%@76 EDP limit 0 RAM 1489/3994MB (lfb 412x4MB) cpu [6%,3%,6%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 1489/3994MB (lfb 412x4MB) cpu [6%,2%,3%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1489/3994MB (lfb 412x4MB) cpu [7%,4%,3%,8%]@102 GR3D 0%@76 EDP limit 0 RAM 1489/3994MB (lfb 412x4MB) cpu [6%,2%,1%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 1489/3994MB (lfb 412x4MB) cpu [8%,4%,1%,4%]@102 GR3D 0%@76 EDP limit 0
- Tegrastats with the above pipeline running
RAM 1785/3994MB (lfb 408x4MB) cpu [45%,44%,51%,40%]@1428 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [47%,53%,50%,45%]@1036 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [46%,54%,42%,47%]@1326 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [44%,45%,47%,49%]@1036 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [46%,50%,48%,47%]@1036 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [48%,48%,49%,45%]@1224 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [50%,50%,43%,46%]@1326 GR3D 0%@76 EDP limit 0 RAM 1786/3994MB (lfb 408x4MB) cpu [50%,47%,48%,43%]@1326 GR3D 0%@76 EDP limit 0 RAM 1785/3994MB (lfb 408x4MB) cpu [40%,46%,47%,46%]@1132 GR3D 0%@76 EDP limit 0 RAM 1786/3994MB (lfb 408x4MB) cpu [46%,53%,40%,49%]@1428 GR3D 0%@76 EDP limit 0
Sextuple Capture, and Display
Using the following pipelines we can test the performance of the Jetson TX1 when doing sextuple video capture and display:
V4l2src
V4l2 can't be used with the ISP accelerated hardware unit of the Jetson TX1 to do the de-Bayer process (convert to YUV color-space) yet. Because of that, we can't link v4l2src with any hardware-accelerated encoder/decoder available on Jetson TX1 or any actual video sinks, if the camera sensor output is in Bayer color-space format.
Nvcamerasrc
- Pipeline for sextuple video capture, at 1920x1080 resolution @30fps and display:
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink nvcamerasrc sensor-id=1 fpsRange="30 30" ! \ 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! \ nveglglessink nvcamerasrc sensor-id=2 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink nvcamerasrc sensor-id=3 fpsRange="30 30" ! \ 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! \ nveglglessink nvcamerasrc sensor-id=4 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink nvcamerasrc sensor-id=5 fpsRange="30 30" ! \ 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! perf ! \ nvegltransform ! nveglglessink -e
Performance statistics
- Tegrastats in normal operation:
RAM 1661/3994MB (lfb 334x4MB) cpu [3%,7%,5%,1%]@102 GR3D 0%@76 EDP limit 0 RAM 1661/3994MB (lfb 334x4MB) cpu [2%,1%,2%,1%]@102 GR3D 0%@76 EDP limit 0 RAM 1660/3994MB (lfb 334x4MB) cpu [5%,2%,1%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 1660/3994MB (lfb 334x4MB) cpu [3%,3%,4%,6%]@102 GR3D 0%@76 EDP limit 0 RAM 1660/3994MB (lfb 334x4MB) cpu [3%,4%,1%,7%]@102 GR3D 0%@76 EDP limit 0 RAM 1660/3994MB (lfb 334x4MB) cpu [5%,3%,1%,6%]@102 GR3D 0%@76 EDP limit 0 RAM 1661/3994MB (lfb 334x4MB) cpu [3%,3%,1%,9%]@102 GR3D 0%@76 EDP limit 0 RAM 1661/3994MB (lfb 334x4MB) cpu [3%,4%,3%,3%]@102 GR3D 0%@76 EDP limit 0 RAM 1661/3994MB (lfb 334x4MB) cpu [3%,2%,3%,4%]@102 GR3D 0%@76 EDP limit 0 RAM 1661/3994MB (lfb 334x4MB) cpu [2%,4%,4%,6%]@102 GR3D 0%@76 EDP limit 0
- Tegrastats with the above pipeline running
RAM 2253/3994MB (lfb 284x4MB) cpu [63%,63%,64%,61%]@1734 GR3D 52%@307 EDP limit 0 RAM 2253/3994MB (lfb 284x4MB) cpu [63%,64%,60%,61%]@1734 GR3D 59%@230 EDP limit 0 RAM 2252/3994MB (lfb 284x4MB) cpu [64%,64%,64%,61%]@1734 GR3D 61%@230 EDP limit 0 RAM 2252/3994MB (lfb 284x4MB) cpu [65%,58%,64%,60%]@1734 GR3D 65%@307 EDP limit 0 RAM 2252/3994MB (lfb 284x4MB) cpu [61%,67%,62%,64%]@1734 GR3D 54%@307 EDP limit 0 RAM 2252/3994MB (lfb 284x4MB) cpu [66%,58%,63%,64%]@1734 GR3D 78%@230 EDP limit 0 RAM 2254/3994MB (lfb 283x4MB) cpu [65%,64%,64%,63%]@1734 GR3D 57%@307 EDP limit 0 RAM 2254/3994MB (lfb 283x4MB) cpu [65%,64%,62%,64%]@1734 GR3D 51%@230 EDP limit 0 RAM 2254/3994MB (lfb 283x4MB) cpu [67%,59%,63%,58%]@1734 GR3D 73%@230 EDP limit 0 RAM 2254/3994MB (lfb 283x4MB) cpu [62%,69%,67%,61%]@1734 GR3D 45%@307 EDP limit 0
Sextuple Capture, Downscale and Display
Using the following pipelines we can test the performance of the Jetson TX1 when doing sextuple video capture, downscale, and display:
V4l2src
V4l2 can't be used with the ISP accelerated hardware unit of the Jetson TX1 to do the de-Bayer process (convert to YUV color-space) yet. Because of that, we can't link v4l2src with any hardware-accelerated encoder/decoder available on Jetson TX1 or any actual video sinks, if the camera sensor output is in Bayer color-space format.
Nvcamerasrc
This was the same pipeline used in the ov5647 sextuple capture on the Jetson TX1 demo video in the Overview Video section at the beginning of this wiki.
- Pipeline for sextuple video capture, downscale from 1920x1080 to 640x480 resolution @30fps and display:
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, \ framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=1 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, \ height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, \ format=(string)I420, framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=2 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \ width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, \ height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=3 fpsRange="30 30" ! \ 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! \ 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=4 \ fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \ nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! xvimagesink \ nvcamerasrc sensor-id=5 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \ framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, \ framerate=(fraction)30/1' ! xvimagesink -v
The following picture is a screenshot of the Jetson TX1 when it was running the above pipeline:
Performance statistics
- Tegrastats in normal operation:
RAM 1694/3994MB (lfb 351x4MB) cpu [4%,3%,1%,8%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [4%,3%,3%,7%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [4%,4%,3%,6%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [4%,5%,0%,7%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [1%,4%,0%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [7%,5%,1%,5%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [8%,3%,1%,2%]@518 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [3%,0%,0%,0%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [3%,1%,0%,0%]@102 GR3D 0%@76 EDP limit 0 RAM 1694/3994MB (lfb 351x4MB) cpu [4%,3%,1%,0%]@102 GR3D 0%@76 EDP limit 0
- Tegrastats with the above pipeline running
RAM 2065/3994MB (lfb 340x4MB) cpu [68%,57%,60%,59%]@1734 GR3D 54%@76 EDP limit 0 RAM 2065/3994MB (lfb 340x4MB) cpu [66%,59%,56%,57%]@1734 GR3D 64%@76 EDP limit 0 RAM 2065/3994MB (lfb 340x4MB) cpu [60%,65%,58%,62%]@1734 GR3D 53%@76 EDP limit 0 RAM 2065/3994MB (lfb 340x4MB) cpu [60%,62%,60%,63%]@1734 GR3D 46%@76 EDP limit 0 RAM 2065/3994MB (lfb 340x4MB) cpu [60%,58%,60%,65%]@1734 GR3D 54%@76 EDP limit 0 RAM 2065/3994MB (lfb 340x4MB) cpu [61%,64%,62%,60%]@1734 GR3D 42%@76 EDP limit 0 RAM 2065/3994MB (lfb 340x4MB) cpu [63%,61%,59%,65%]@1734 GR3D 54%@76 EDP limit 0 RAM 2069/3994MB (lfb 339x4MB) cpu [62%,63%,60%,58%]@1734 GR3D 51%@76 EDP limit 0 RAM 2069/3994MB (lfb 339x4MB) cpu [63%,57%,62%,59%]@1734 GR3D 57%@76 EDP limit 0 RAM 2069/3994MB (lfb 339x4MB) cpu [61%,66%,64%,62%]@1734 GR3D 56%@76 EDP limit 0
Video Encoding Transport Stream 1920x1080@30fps
The following pipeline will generate a video, you can visualize it with any video-player like VLC for example.
gst-launch-1.0 -v nvcamerasrc sensor-id=1 fpsRange="30 30" num-buffers=500 ! 'video/x-raw(memory:NVMM), width=(int)1920, \ height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh264enc ! qtmux ! filesink location=test.ts
OV5647 4x camera 720p recording test
On OV5647_4x_camera_720p_recording_test you will find a very descriptive step by step 4x camera 720p video stream recording to disk test. The test consists in save to disk the 4 video streams in RAW, H264, and H265 encoded formats in each case. A SSD and a SD Card (class 10) were used as disk units. Also, you will find the performance statistics of each test case, so you can make comparisons between them.
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.
ARTIFICIAL INTELLIGENCE
