Jump to content

GStreamer pipelines for Jetson TX2: Difference between revisions

GStreamer pipelines for Jetson TX2 (view source)

Revision as of 13:02, 7 June 2017

17,049 bytes added ,  7 June 2017
→‎Articles related
Line 913: Line 913:
*Latency=251ms
*Latency=251ms
<br>
<br>
= Articles related =
= nvgstcapture-1.0 =
<br>
 
This is an application based on gstreamer and omx to capture, encode and save video to the filesystem. It has a lot of options. The source code is not released by NVIDIA yet but they are planning to release in future versions of Linx 4 Tegra (L4T), you can read about it at [https://devtalk.nvidia.com/default/topic/898129/jetson-tx1/enabling-camera-on-jetson-tx1-board/ Nvidia Embedded Computing Forum]
 
<pre>
nvidia@tegra-ubuntu:~$ nvgstcapture-1.0 --help
Encoder null, cannot set bitrate!
Encoder Profile = Baseline
Usage:
  nvgstcapture-1.0 [OPTION...] Nvidia GStreamer Camera Model Test
 
Help Options:
  -h, --help                        Show help options
  --help-all                        Show all help options
  --help-gst                        Show GStreamer Options
 
Application Options:
  --prev-res                        Preview width & height. USB Range: 0 to 12 (5632x4224) and CSI Range: 2 to 12 (5632x4224) e.g., --prev-res=3
  --cus-prev-res                    Custom Preview width & height [for CSI only] e.g., --cus-prev-res=1920x1080
  --image-res                        Image width & height. Range: 0 to 12 (5632x4224) e.g., --image-res=3
  --video-res                        Video width & height. Range: 0 to 9 (3896x2192) e.g., --video-res=3
  -m, --mode                        Capture mode value (1=still 2=video)
  -v, --video-enc                    Video encoder type (0=h264[HW] 1=vp8[HW] 2=h265[HW] 3=vp9[HW])
  -b, --enc-bitrate                  Video encoding Bit-rate(in bytes) e.g., --enc-bitrate=4000000
  --enc-profile                      Video encoder profile (Only for H.264) (0=Baseline, 1=Main, 2=High)
  -J, --image-enc                    Image encoder type (0=jpeg_SW[jpegenc] 1=jpeg_HW[nvjpegenc])
  -k, --file-type                    Container file type (0=mp4 1=3gp 2=mkv)
  --cap-dev-node                    Video capture device node (0=/dev/video0[default], 1=/dev/video1, 2=/dev/video2) e.g., --cap-dev-node=0
  --svs                              [For USB] (=) chain for video Preview. [For CSI only] use "nvoverlaysink"
  --file-name                        Captured file name. nvcamtest is used by default
  --camsrc                          Camera Source to use (0=v4l2, 1=csi[default], 2=videotest, 3=eglstream)
  --orientation                      [For CSI only] Camera sensor orientation value
  -w, --whitebalance                [For CSI only] Capture whitebalance value
  -s, --scene-mode                  [For CSI only] Camera Scene-Mode value
  -c, --color-effect                [For CSI only] Camera Color Effect value
  --auto-exposure                    [For CSI only] Camera Auto-Exposure value
  --flash                            [For CSI only] Camera Flash value
  --flicker                          [For CSI only] Camera Flicker Detection and Avoidance Mode value
  --contrast                        [For CSI only] Camera Contrast value
  --saturation                      [For CSI only] Camera Saturation value
  --edge-enhancement                [For CSI only] Camera Edge Enhancement value
  --tnr-strength                    [For CSI only] Camera TNR-Strength value
  --tnr-mode                        [For CSI only] Camera TNR Mode value
  --sensor-id                        [For CSI only] Camera Sensor ID value
  --eglstream-id                    [For CSI EGLStream Consumer] Select EGLStream Producer ID value. Default value 0
  --display-id                      [For nvoverlaysink only] Display ID value
  --aeRegion                        [For CSI Only] ROI for AE coordinates(top,left,bottom,right) and weight in that order.e.g., --aeRegion="30 40 200 200 1.2"
  --wbRegion                        [For CSI Only] ROI for AWB coordinates(top,left,bottom,right) and weight in that order.e.g., --wbRegion="30 40 200 200 1.2"
  --fpsRange                        [For CSI Only] FPS range values (low, high) e.g., --fpsRange="15 30"
  --exposure-time                    [For CSI only] Capture exposure time value. e.g., --exposure-time=0.033
  --wbGains                          [For CSI Only] WB gains values (R, GR, GB, B) in that order. e.g., --wbGains="1.2 1.4 0.8 1.6"
  --overlayConfig                    Overlay Configuration Options index and coordinates in (index, x_pos, y_pos, width, height) order  e.g. --overlayConfig="0, 0, 0, 1280, 720"
  --enable-exif                      Enable Exif data
  --dump-bayer                      Dump bayer data in addition to image capture.
  --eglConfig                        EGL window Coordinates (x_pos y_pos) in that order  e.g., --eglConfig="50 100"
  --aeLock                          [For CSI only] Enable auto exposure lock e.g., --aeLock
  --enable-meta                      Enable Sensor MetaData reporting
  --nvvideosink-create-eglstream    [For nvvideosink EGLStream Producer] Enable nvvideosink EGLStream Producer
  --callback                        Enable User callback to set capture settings per frame
  --app-profile                      Enable KPI profiling
  --kpi-numbers                      Enable KPI measurement
  -A, --automate                    Run application in automation mode
  -S, --start-time                  Start capture after specified time in seconds. Default = 5 sec (use with --automate or -A only)
  -Q, --quit-after                  Quit application once automation is done after specified time in seconds. Default = 0 sec (use with --automate or -A only)
  -C, --count                        Number of iterations of automation testcase. Default = 1 (use with --automate or -A only)
  --capture-gap                      Number of milliseconds between successive image/video capture. Default = 250 msec (use with --automate and --capture-auto only)
  --capture-time                    Capture video for specified time in seconds. Default = 10 sec (use with --automate and --capture-auto only)
  --toggle-mode                      Toggle between still and video capture modes for count number of times (use with --automate or -A only)
  --toggle-sensor                    Toggle between sensor-id 0 and 1 (use with --automate or -A only)
  --capture-auto                    Do image/video capture in automation mode for count number of times(use with --automate or -A only)
  --enum-wb                          Enumerate all white-balance modes for count number of times (use with --automate or -A only)
  --enum-scm                        Enumerate all scene modes for count number of times (use with --automate or -A only)
  --enum-ce                          Enumerate all color-effect modes for count number of times (use with --automate or -A only)
  --enum-ae                          Enumerate all auto-exposure modes for count number of times (use with --automate or -A only)
  --enum-f                          Enumerate all flash modes for count number of times (use with --automate or -A only)
  --enum-fl                          Enumerate all flicker detection and avoidance modes for count number of times (use with --automate or -A only)
  --enum-ct                          Enumerate contrast value through 0 to 1 by a step of 0.1 for count number of times (use with --automate or -A only)
  --enum-st                          Enumerate saturation value through 0 to 2 by a step of 0.1 for count number of times (use with --automate or -A only)
  --enum-ee                          Enumerate edge-enhancement value through 0 to 1 by a step of 0.1 for count number of times (use with --automate or -A only)
  --enum-ts                          Enumerate TNR strength value through 0 to 1 by a step of 0.1 for count number of times (use with --automate or -A only)
  --enum-tnr                        Enumerate all TNR modes for count number of times (use with --automate or -A only)
  --streaming-mode                  [For CSI Only] Enable streaming mode (1=stream only, 2=stream+record)
  --streaming-file                  File to stream instead of camera input
 
Supported resolutions in case of CSI Camera
  (2) : 640x480
  (3) : 1280x720
  (4) : 1920x1080
  (5) : 2104x1560
  (6) : 2592x1944
  (7) : 2616x1472
  (8) : 3840x2160
  (9) : 3896x2192
  (10): 4208x3120
  (11): 5632x3168
  (12): 5632x4224
 
Runtime CSI Camera Commands:
 
  Help : 'h'
  Quit : 'q'
  Set Capture Mode:
      mo:<val>
          (1): image
          (2): video
  Get Capture Mode:
      gmo
  Set Sensor Id (0 to 10):
      sid:<val> e.g., sid:2
  Get Sensor Id:
      gsid
  Set sensor orientation:
      so:<val>
          (0): none
          (1): Rotate counter-clockwise 90 degrees
          (2): Rotate 180 degrees
          (3): Rotate clockwise 90 degrees
  Get sensor orientation:
      gso
  Set Whitebalance Mode:
      wb:<val>
          (0): off
          (1): auto
          (2): incandescent
          (3): fluorescent
          (4): warm-fluorescent
          (5): daylight
          (6): cloudy-daylight
          (7): twilight
          (8): shade
          (9): manual
  Get Whitebalance Mode:
      gwb
  Set Scene-Mode:
      scm:<val>
          (0): face-priority
          (1): action
          (2): portrait
          (3): landscape
          (4): night
          (5): night-portrait
          (6): theatre
          (7): beach
          (8): snow
          (9): sunset
          (10): steady-photo
          (11): fireworks
          (12): sports
          (13): party
          (14): candle-light
          (15): barcode
  Get Scene-Mode:
      gscm
  Set Color Effect Mode:
      ce:<val>
          (1): off
          (2): mono
          (3): negative
          (4): solarize
          (5): sepia
          (6): posterize
          (7): aqua
  Get Color Effect Mode:
      gce
  Set Auto-Exposure Mode:
      ae:<val>
          (1): off
          (2): on
          (3): OnAutoFlash
          (4): OnAlwaysFlash
          (5): OnFlashRedEye
  Get Auto-Exposure Mode:
      gae
  Set Flash Mode:
      f:<val>
          (0): off
          (1): on
          (2): torch
          (3): auto
  Get Flash Mode:
      gf
  Set Flicker Detection and Avoidance Mode:
      fl:<val>
          (0): off
          (1): 50Hz
          (2): 60Hz
          (3): auto
  Get Flicker Detection and Avoidance Mode:
      gfl
  Set Contrast (0 to 1):
      ct:<val> e.g., ct:0.75
  Get Contrast:
      gct
  Set Saturation (0 to 2):
      st:<val> e.g., st:1.25
  Get Saturation:
      gst
  Set Exposure Time in seconds:
      ext:<val> e.g., ext:0.033
  Get Exposure Time:
      gext
  Set Auto Exposure Lock(0/1):
      ael:<val> e.g., ael:1
  Get Auto Exposure Lock:
      gael
  Set Edge Enhancement (0 to 1):
      ee:<val> e.g., ee:0.75
  Get Edge Enhancement:
      gee
  Set ROI for AE:
      It needs five values, ROI coordinates(top,left,bottom,right)
      and weight in that order
      aer:<val> e.g., aer:20 20 400 400 1.2
  Get ROI for AE:
      gaer
  Set ROI for AWB:
      It needs five values, ROI coordinates(top,left,bottom,right)
      and weight in that order
      wbr:<val> e.g., wbr:20 20 400 400 1.2
  Get ROI for AWB:
      gwbr
  Set FPS range:
      It needs two values, FPS Range (low, high) in that order
      fpsr:<val> e.g., fpsr:15 30
  Get FPS range:
      gfpsr
  Set WB Gains:
      It needs four values (R, GR, GB, B) in that order
      wbg:<val> e.g., wbg:1.2 2.2 0.8 1.6
  Get WB Gains:
      gwbg
  Set TNR Strength (0 to 1):
      ts:<val> e.g., ts:0.75
  Get TNR Strength:
      gts
  Set TNR Mode:
      tnr:<val>
          (0): NoiseReduction_Off
          (1): NoiseReduction_Fast
          (2): NoiseReduction_HighQuality
  Get TNR Mode:
      gtnr
  Capture: enter 'j' OR
          followed by a timer (e.g., jx5000, capture after 5 seconds) OR
          followed by multishot count (e.g., j:6, capture 6 images)
          timer/multihot values are optional, capture defaults to single shot with timer=0s
  Start Recording : enter '1'
  Stop Recording  : enter '0'
  Video snapshot : enter '2' (While recording video)
  Set Preview Resolution:
      pcr:<val> e.g., pcr:3
          (2) : 640x480
          (3) : 1280x720
          (4) : 1920x1080
          (5) : 2104x1560
          (6) : 2592x1944
          (7) : 2616x1472
          (8) : 3840x2160
          (9) : 3896x2192
          (10): 4208x3120
          (11): 5632x3168
          (12): 5632x4224
  Note - For preview resolutions 4208x3120 and more use option --svs=nveglglessink
  Get Preview Resolution:
      gpcr
  Set Image Resolution:
      icr:<val> e.g., icr:3
          (2) : 640x480
          (3) : 1280x720
          (4) : 1920x1080
          (5) : 2104x1560
          (6) : 2592x1944
          (7) : 2616x1472
          (8) : 3840x2160
          (9) : 3896x2192
          (10): 4208x3120
          (11): 5632x3168
          (12): 5632x4224
  Get Image Capture Resolution:
      gicr
  Set Video Resolution:
      vcr:<val> e.g., vcr:3
          (2) : 640x480
          (3) : 1280x720
          (4) : 1920x1080
          (5) : 2104x1560
          (6) : 2592x1944
          (7) : 2616x1472
          (8) : 3840x2160
          (9) : 3896x2192
  Get Video Capture Resolution:
      gvcr
 
Runtime USB Camera Commands:
 
  Help : 'h'
  Quit : 'q'
  Set Capture Mode:
      mo:<val>
          (1): image
          (2): video
  Get Capture Mode:
      gmo
  Capture: enter 'j' OR
          followed by a timer (e.g., jx5000, capture after 5 seconds) OR
          followed by multishot count (e.g., j:6, capture 6 images)
          timer/multihot values are optional, capture defaults to single shot with timer=0s
  Start Recording : enter '1'
  Stop Recording  : enter '0'
  Set Preview Resolution:
      pcr:<val> e.g., pcr:2
          (0) : 176x144
          (1) : 320x240
          (2) : 640x480
          (3) : 1280x720
          (4) : 1920x1080
  NOTE: Preview/Encode resolution will be same as Capture resolution for USB-Camera
  Get Preview Resolution:
      gpcr
  Get Image Capture Resolution:
      gicr
  Get Video Capture Resolution:
      gvcr
  Set Capture Device Node:
      cdn:<val> e.g., cdn:0
          (0): /dev/video0
          (1): /dev/video1
          (2): /dev/video2
  Get Capture Device Node:
      gcdn
 
Runtime encoder configuration options:
 
  Set Encoding Bit-rate(in bytes):
      br:<val> e.g., br:4000000
  Get Encoding Bit-rate(in bytes):
      gbr
  Set Encoding Profile(only for H.264):
      ep:<val> e.g., ep:1
          (0): Baseline
          (1): Main
          (2): High
  Get Encoding Profile(only for H.264):
      gep
  Force IDR Frame on video Encoder(only for H.264):
      Enter 'f'
 
</pre>
 
== Capture snapshots ==
 
Sometimes when the board has been just flashed you need to run reboot in order to run nvgstcapture-1.0 successfully. To take snapshots:
 
<pre>
nvgstcapture-1.0
#then press j to take an snapshot
q
</pre>
 
Resolutions available to create snapshot:
<pre>
          (2) : 640x480
          (3) : 1280x720
          (4) : 1920x1080
          (5) : 2104x1560
          (6) : 2592x1944
          (7) : 2616x1472
          (8) : 3840x2160
          (9) : 3896x2192
          (10): 4208x3120
          (11): 5632x3168
          (12): 5632x4224
 
</pre>
 
<pre>
nvgstcapture-1.0 -m 1 --image-res=10 #set 1 to snapshot mode
</pre>
 
In general the application is consuming 15% of each of four cores. It is always displaying video this could affect.
 
== Capture and record video ==
 
<pre>
nvgstcapture-1.0 -m 2 # set the mode in video
1 # start video recording
2 # take a snapshot
0 # stop video recording
 
</pre>
 
In case of video it should support:
<pre>
          (2) : 640x480
          (3) : 1280x720
          (4) : 1920x1080
          (5) : 2104x1560
          (6) : 2592x1944
          (7) : 2616x1472
          (8) : 3840x2160
          (9) : 3896x2192
</pre>
 
However, it is not possible to set other value than 2,3,4,5 up to 9.
 
<pre>
nvgstcapture-1.0 -m 2 --video-res=5
</pre>
 
'''top'''
<pre>
 
top - 22:15:09 up  2:30,  4 users,  load average: 0.47, 0.45, 0.30
Tasks: 267 total,  1 running, 266 sleeping,  0 stopped,  0 zombie
%Cpu(s): 11.0 us,  6.0 sy,  0.0 ni, 83.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  8041332 total,  754840 free,  1611400 used,  5675092 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  6303032 avail Mem
 
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM    TIME+ COMMAND                                                                                                                                                           
1639 root      20  0  9.979g 264612  45284 S  52.3  3.3  6:14.66 nvcamera-daemon                                                                                                                                                   
27513 nvidia    20  0  815040  13932  10272 S  4.3  0.2  0:01.48 nvgstcapture-1.
 
</pre>
 
There is a daemon and a pipeline. Interesting...
 
While recording the pipeline (nvgstcapture) increases the ARM load to 10% and then it goes down again, even with the highest resolution. It will be interesting to understand what is the daemon doing.
 
= How to Contact =
= How to Contact =


Retrieved from "https://developer.ridgerun.com/wiki/index.php/Special:MobileDiff/12633"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.