Birds Eye View - Generic Pipelines

RidgeRun's generic Bird's Eye View GStreamer pipelines show how to generate a top-down surround-view output on standard Linux systems such as x86 development PCs. Use this page when you want copy-pasteable gst-launch-1.0 examples for JPEG images, MP4 files, UDP/RTP streams, or live V4L2 cameras. These examples assume that you already have a valid calibration file and that the input order in the pipeline matches the camera order used during calibration.

This page focuses on generic Linux pipelines that use software decode and display paths. If you need hardware-accelerated pipelines for NVIDIA Jetson or NXP i.MX8, use Birds Eye View/GStreamer/Jetson Pipelines or Birds Eye View/GStreamer/i.MX8 Pipelines. For the calibration workflow, see Birds Eye View/Calibration Guide. For the BEV plugin interface and supported element properties, see Birds Eye View/GStreamer/GstBEV Plugin.

Table 1 summarizes the example pipelines provided in this page as a reference starting point:

Table 1: Choose the right pipeline
Input source	Output example	Best for
JPEG images	Single JPEG result	Functional validation of calibration and source ordering
UDP/RTP H.264 streams	MP4 file or UDP/RTP stream	Multi-camera network ingest and recording
MP4 files	Local display	Repeatable offline testing with recorded inputs
Live V4L2 cameras	Local display	End-to-end camera bring-up on generic Linux

Set up the Environment

Create or locate the calibration file first. For the examples below, we use an example calibration file included with the evaluation binaries which assumes a six-camera setup. Adjust the number of sources accordingly in your pipelines. We set the calibration file and the input image caps as environment variables as follows:

CALIBRATION_FILE=birds_eye_view.json 
CAPS="video/x-raw,format=RGBA"

You'll notice that we deliberately don't specify the resolution in the caps, since it should be automatically picked up by the codecs.

Generate Birds Eye View from JPEG images

In this example, the pipeline will read the images presented in Fig. 1 and generate one single birds eye view image.

Fig. 1: Example input images for generating birds eye view
Camera 0
Camera 1
Camera 2
Camera 3
Camera 4
Camera 5

Note that there is one filesrc branch per camera image in the pipeline, all images are sent to the bev element and the output is stored as a jpeg image.

gst-launch-1.0 bev name=bev0 calibration-file=$CALIBRATION_FILE \
 filesrc location=samples/bev_6_cameras/cam_0.jpg ! jpegparse ! jpegdec ! videoconvert ! queue ! bev0.sink_0 \
 filesrc location=samples/bev_6_cameras/cam_1.jpg ! jpegparse ! jpegdec ! videoconvert ! queue ! bev0.sink_1 \
 filesrc location=samples/bev_6_cameras/cam_2.jpg ! jpegparse ! jpegdec ! videoconvert ! queue ! bev0.sink_2 \
 filesrc location=samples/bev_6_cameras/cam_5.jpg ! jpegparse ! jpegdec ! videoconvert ! queue ! bev0.sink_3 \
 filesrc location=samples/bev_6_cameras/cam_4.jpg ! jpegparse ! jpegdec ! videoconvert ! queue ! bev0.sink_4 \
 filesrc location=samples/bev_6_cameras/cam_3.jpg ! jpegparse ! jpegdec ! videoconvert ! queue ! bev0.sink_5 \
 bev0. ! queue ! videoconvert ! jpegenc ! filesink location=bev_result.jpg

The expected output from the previous pipeline is presented in Fig. 2.

Generate Birds Eye view from UDP video streams

This section provides pipelines for reading the input video stream from UDP and saving the resulting birds eye view video as an MP4 recording or streaming it through UDP. The examples in this section assume that the input video streams are available in the UDP ports from 4000 to 4005.

Save the result to a MP4 file

The pipeline below receives six UDP streams, decodes them, uses the bev element to generate a single top-down view, encodes the result and saves it as a MP4 file.

gst-launch-1.0 -ve \
udpsrc port=4000 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_0 \
udpsrc port=4001 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_1 \
udpsrc port=4002 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_2 \
udpsrc port=4003 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_3 \
udpsrc port=4004 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_4 \
udpsrc port=4005 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_5 \
bev name=bev calibration-file=$CALIBRATION_FILE ! queue ! videoconvert ! x264enc ! h264parse ! mp4mux ! filesink location=bev_result.mp4

Stream the result via UDP+RTP

Similarly to the previous example, the following pipeline receives six UDP video streams and generates the birds eye view. The resulting video stream is then encoded and streamed through UDP. You must set the HOST to the IP address and PORT to the UDP port of the birds eye view video receiver:

Sender

gst-launch-1.0 -v \
udpsrc port=4000 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_0 \
udpsrc port=4001 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_1 \
udpsrc port=4002 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_2 \
udpsrc port=4003 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_3 \
udpsrc port=4004 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_4 \
udpsrc port=4005 ! application/x-rtp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! queue ! bev.sink_5 \
bev name=bev calibration-file=$CALIBRATION_FILE ! queue ! videoconvert ! x264enc ! rtph264pay config-interval=10  ! queue ! udpsink host=$HOST port=$PORT

You can use the following pipeline to receive and display the birds eye view stream:

Receiver

gst-launch-1.0 udpsrc port=$PORT ! 'application/x-rtp,media=(string)video,encoding-name=(string)H264' !  queue ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink

Generate Birds Eye view from mp4 videos

The following pipeline reads the input from six mp4 videos, decodes them and applies the birds eye view. The result is sent to display. Make sure to set the INPUT_N variables to your corresponding input videos paths.

gst-launch-1.0 bev name=bev0 calibration-file=$CALIBRATION_FILE \
 filesrc location=$INPUT_0 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! $CAPS ! bev0.sink_0 \
 filesrc location=$INPUT_1 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert !  $CAPS ! bev0.sink_1 \
 filesrc location=$INPUT_2 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert !  $CAPS ! bev0.sink_2 \
 filesrc location=$INPUT_3 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert !  $CAPS ! bev0.sink_3 \
 filesrc location=$INPUT_4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert !  $CAPS ! bev0.sink_4 \
 filesrc location=$INPUT_5 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert !  $CAPS ! bev0.sink_5 \
 bev0. !  queue ! videoconvert ! xvimagesink

Generate Birds Eye View from live cameras

The following pipeline captures from six cameras, applies the birds eye view and sends the result to display. Make sure to set the video device ID and the caps property according to your own setup.

# We need to define the desired resolution here since v4l2src can give us various image sizes.
CAPS="video/x-raw,width=1280,height=720,format=RGBA"

gst-launch-1.0 bev name=bev0 calibration-file=$CALIBRATION_FILE \
 v4l2src device=/dev/video0 ! videoconvert ! $CAPS ! bev0.sink_0 \
 v4l2src device=/dev/video2 ! videoconvert ! $CAPS ! bev0.sink_1 \
 v4l2src device=/dev/video4 ! videoconvert ! $CAPS ! bev0.sink_2 \
 v4l2src device=/dev/video6 ! videoconvert ! $CAPS ! bev0.sink_3 \
 v4l2src device=/dev/video8 ! videoconvert ! $CAPS ! bev0.sink_4 \
 v4l2src device=/dev/video10 ! videoconvert ! $CAPS ! bev0.sink_5 \
 bev0. !  queue ! videoconvert ! autovideosink

Performance considerations

These generic pipelines are intended for functional validation and generic Linux integration. They are not optimized for every production target. Performance depends on:

Number of cameras
Input resolution and frame rate
Codec complexity and encoder settings
CPU and memory bandwidth
Linux distribution, kernel, and GStreamer versions
Whether decode, colorspace conversion, and display are handled in software

If you need platform-optimized pipelines, move to Birds Eye View/GStreamer/Jetson Pipelines or Birds Eye View/GStreamer/i.MX8 Pipelines.

FAQ

Do I need a calibration file before running these pipelines?: Yes. The BEV element needs a calibration file that matches the camera count, source order, and input resolution. See Birds Eye View/Calibration Guide.
Can I use these pipelines on Jetson or i.MX8?: You can use them for conceptual reference, but RidgeRun provides dedicated platform-specific pages for accelerated pipelines on Birds Eye View/GStreamer/Jetson Pipelines and Birds Eye View/GStreamer/i.MX8 Pipelines.
How do I confirm that my Bird's Eye View output is correct?: Start with the JPEG example, compare the result against a known-good stitched image, and then verify camera order, calibration data, and input caps before moving to live or networked sources.

References

⇦ GStreamer/GstBEV Plugin

Home

GStreamer/Jetson Pipelines ⇨

❯

Share This Page

Send Email Using