NVIDIA Jetson AGX Thor - H264 GStreamer Pipelines
The NVIDIA Jetson AGX Thor documentation from RidgeRun is presently being developed. |
GStreamer Pipelines using H.264
This section presents GStreamer pipelines to demonstrate H.264 encoding and decoding on Jetson AGX Thor. The examples go from raw video playback and software encoding, to full camera capture pipelines with hardware acceleration. Performance metrics are provided for each case, tested under multiple power modes.
For benchmarking methodology, see GStreamer Pipelines Overview.
H.264 Encoding
H.264 (also known as AVC) is a widely used video compression format known for its high compression efficiency and low-latency support. Jetson AGX Thor supports H.264 encoding via NVIDIA hardware encoders as well as a CPU-based encoder.
Available Encoders
Feature | nvv4l2h264enc | nvautogpuh264enc | nvcudah264enc | nvh264enc | x264enc (CPU) |
---|---|---|---|---|---|
Backend | V4L2 (NVIDIA) | NVCODEC auto GPU | NVCODEC CUDA | NVENC API | libx264 |
GPU Support | yes | yes | yes | yes | no |
Input Memory | NVMM | CUDA, GLMemory | CUDA | CUDA | System |
Input Formats | NV12, I420, Y444, etc. | NV12, Y444 | NV12, Y444 | NV12, YV12, RGBA | I420, YUV420 |
Output Format | byte-stream, AU/NAL | avc/byte-stream | avc/byte-stream | byte-stream | byte-stream |
B-frames | 0-2 | 0-4 | 0-4 | 0-4 | full support |
Notes | Lowest latency | Auto GPU selection | Manual GPU selection | GPU0 only | High CPU usage |
Raw Video Encoding
This section demonstrates encoding synthetic video (no camera involved). Useful for testing encoder behavior and CPU/GPU load.
FILE=/tmp/test.mp4 WIDTH=1920 HEIGHT=1080 FRAMERATE=30/1
With x264enc (CPU)
gst-launch-1.0 videotestsrc is-live=true ! \ "video/x-raw, width=$WIDTH, height=$HEIGHT, framerate=$FRAMERATE" ! \ x264enc tune=zerolatency ! h264parse ! qtmux ! filesink location=$FILE -e
With nvv4l2h264enc
gst-launch-1.0 videotestsrc is-live=true ! \ video/x-raw, width=$WIDTH, height=$HEIGHT, framerate=$FRAMERATE ! \ nvvidconv ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILE -e
With nvautogpuh264enc
gst-launch-1.0 videotestsrc is-live=true ! \ video/x-raw, width=$WIDTH, height=$HEIGHT, framerate=$FRAMERATE ! \ nvautogpuh264enc ! h264parse ! qtmux ! filesink location=$FILE -e
With nvh264enc
gst-launch-1.0 videotestsrc is-live=true ! \ video/x-raw, width=$WIDTH, height=$HEIGHT, framerate=$FRAMERATE ! \ nvh264enc ! h264parse ! qtmux ! filesink location=$FILE -e
Camera Encoding Pipelines
MIPI CSI Camera (IMX676)
WIDTH=3552 HEIGHT=3556 FRAMERATE=64/1 FORMAT=NV12 FILE=/tmp/filename.mp4
With x264enc (CPU)
gst-launch-1.0 nvarguscamerasrc ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ x264enc tune=zerolatency ! h264parse ! qtmux ! filesink location=$FILE -e
With nvv4l2h264enc (HW, NVMM path)
gst-launch-1.0 nvarguscamerasrc ! \ video/x-raw(memory:NVMM),width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILE -e
With nvautogpuh264enc (HW, NVCODEC auto-GPU)
gst-launch-1.0 nvarguscamerasrc ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ nvautogpuh264enc ! h264parse ! qtmux ! filesink location=$FILE -e
With nvh264enc (HW, NVENC)
gst-launch-1.0 nvarguscamerasrc ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ nvh264enc ! h264parse ! qtmux ! filesink location=$FILE -e
USB Camera
WIDTH=1920 HEIGHT=1080 FRAMERATE=30/1 FILE=/tmp/usb_out.mp4
With x264enc (CPU)
gst-launch-1.0 v4l2src device=/dev/video2 ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ x264enc tune=zerolatency ! h264parse ! qtmux ! filesink location=$FILE -e
With nvv4l2h264enc (HW, NVMM path)
gst-launch-1.0 v4l2src device=/dev/video2 ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ nvvidconv ! "video/x-raw(memory:NVMM)" ! \ nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$FILE -e
With nvautogpuh264enc (HW, NVCODEC auto-GPU)
gst-launch-1.0 v4l2src device=/dev/video2 ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ nvautogpuh264enc ! h264parse ! qtmux ! filesink location=$FILE -e
With nvh264enc (HW, NVENC)
gst-launch-1.0 v4l2src device=/dev/video2 ! \ video/x-raw,width=$WIDTH,height=$HEIGHT,framerate=$FRAMERATE ! \ nvh264enc ! h264parse ! qtmux ! filesink location=$FILE -e
Camera over Ethernet (CoE)
Coming soon...
Encoding Performance
Encoder | Backend | Hardware Block | Mode | CPU (%) | GPU 3D (%) | NVENC0 (%) | NVENC1 (%) | FPS | Latency (ms) |
---|---|---|---|---|---|---|---|---|---|
x264enc | libx264 | CPU | Default (120W) | 37.96 | — | — | — | 14.45 | 52.86 |
x264enc | libx264 | CPU | Max Perf | 32.98 | — | — | — | 27.67 | 28.59 |
nvv4l2h264enc | V4L2 (NVIDIA) | NVENC (HW) | Default (120W) | 1.38 | 0.42 | TBD | TBD | 30.61 | 15.08 |
nvv4l2h264enc | V4L2 (NVIDIA) | NVENC (HW) | Max Perf | 0.78 | 0.13 | TBD | TBD | 30.57 | 12.76 |
nvautogpuh264enc | NVCodec (auto) | NVENC (HW) | Default (120W) | 1.71 | 0.49 | TBD | TBD | 30.24 | 4.75 |
nvautogpuh264enc | NVCodec (auto) | NVENC (HW) | Max Perf | 0.93 | 0.36 | TBD | TBD | 30.23 | 3.41 |
nvh264enc | NVENC API | NVENC (HW) | Default (120W) | 1.17 | 1.10 | TBD | TBD | 30.06 | 4.06 |
nvh264enc | NVENC API | NVENC (HW) | Max Perf | 0.68 | 0.33 | TBD | TBD | 30.06 | 2.93 |
H.264 Decoding
Decoders Available
Decoder | Backend | Hardware Accel |
---|---|---|
nvv4l2decoder | V4L2 (NVIDIA) | Yes |
avdec_h264 | libav | No (CPU-based) |
Playback Pipelines
You may use this pipelines to decode the videos recorded with any of the encoding methods of the previous sections.
Hardware Decode with nv3dsink
gst-launch-1.0 filesrc location=/tmp/filename.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nv3dsink
Software Decode with xvimagesink
gst-launch-1.0 filesrc location=/tmp/filename.mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink
Decoding Performance
Decoder | Backend | Hardware Block | Mode | CPU (%) | GPU 3D (%) | NVDEC0 (%) | NVDEC1 (%) | FPS | Latency (ms) |
---|---|---|---|---|---|---|---|---|---|
avdec_h264 | libav | CPU | Default | — | — | ||||
avdec_h264 | libav | CPU | Max Perf | — | — | ||||
nvv4l2decoder | V4L2 (NVIDIA) | NVDEC (HW) | Default | ||||||
nvv4l2decoder | V4L2 (NVIDIA) | NVDEC (HW) | Max Perf |
H.264 Transcoding
In many multimedia applications, video streams must be decoded and then re-encoded. This is common in:
- Transcoding to reduce bitrate or change codec settings
- Real-time processing such as object detection or watermarking
- Pipeline compatibility adjustments for container formats or playback devices
Pipelines
The following pipelines demonstrate decoding and encoding using various decoder and encoder elements available on Jetson AGX Thor. All pipelines assume the source is a H.264-encoded MP4 file.
Set up common variables:
FILE=/tmp/filename.mp4 OUTFILE=/tmp/reencoded.mp4
nvv4l2decoder + nvv4l2h264enc
gst-launch-1.0 filesrc location=$FILE ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=$OUTFILE -e
nvv4l2decoder + nvh264enc
gst-launch-1.0 filesrc location=$FILE ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw ! nvh264enc ! h264parse ! qtmux ! filesink location=$OUTFILE -e
nvh264dec + nvh264enc
gst-launch-1.0 filesrc location=$FILE ! qtdemux ! h264parse ! nvh264dec ! nvh264enc ! h264parse ! qtmux ! filesink location=$OUTFILE -e
avdec_h264 + x264enc (CPU-only)
gst-launch-1.0 filesrc location=$FILE ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! x264enc ! h264parse ! qtmux ! filesink location=$OUTFILE -e
Plugin Reference Links
- NVIDIA V4L2 plugins (e.g., nvv4l2h264enc, nvv4l2decoder, nvvidconv, nv3dsink): Linux Multimedia API – GStreamer Plugins
- NVCodec/NVENC plugins (e.g., nvh264enc, nvh264dec, nvautogpuh264enc, nvcudah264enc): GStreamer NVCodec Plugin
- x264enc (CPU): GStreamer x264enc
- avdec_h264 (CPU): GStreamer avdec_h264
- h264parse: GStreamer h264parse
- qtdemux: GStreamer qtdemux
- videoconvert: GStreamer videoconvert