GStreamer Support on Rubik Pi 3
🚧 Documentation under development
The Rubik Pi 3 guide is currently under active development. Some sections may be incomplete or change without notice.
Questions? Contact RidgeRun or email to support@ridgerun.com.
RUBIK Pi 3 is a strong GStreamer platform because the board combines camera interfaces, GPU-backed transforms, AI-oriented plugins and network I/O capabilities to build preview, record, inference, and streaming pipelines on one board. For most developers, GStreamer is the fastest way to turn RUBIK Pi 3 into a multimedia capable system. This page provides validated GStreamer pipelines for the Rubik Pi 3. All the pipelines provided in this wiki were tested in an Ubuntu 24.04.4 LTS image with GStreamer 1.24.2.
This page is part of Rubik Pi 3. It focuses on camera capture, transforms, display, recording, AI-assisted multimedia, and RidgeRun GStreamer integration on the QCS6490-based board.
Why GStreamer matters on RUBIK Pi 3
GStreamer matters because most edge AI products are also camera products, display products, record / stream products, or robotics products that happen to include inference. GStreamer provides a graph-based way to compose those stages cleanly.
qtiqmmfsrcfor camera capture,qtivtransformfor accelerated transform operations such as color conversion, crop, and resize,qtimltflitefor TensorFlow Lite model execution on the NPU in Qdemo-style workflows,v4l2h264encfor hardware accelerated H.264 encoding,- and standard GStreamer sinks, encoders, muxers, and utility elements.
How to enable GStreamer on Ubuntu
This step requires the board to have internet connection, see how to connect to the network here. Install the GStreamer pre-built packages with the following commands:
git clone -b ubuntu_setup --single-branch https://github.com/rubikpi-ai/rubikpi-script.git cd rubikpi-script ./install_ppa_pkgs.sh
Use the following command to verify the Qualcomm plugins are installed:
gst-inspect-1.0 | grep qti
This is the output expected in a Ubuntu 24.04.4 LTS image:
qtibatch: qtibatch: Batching stream buffers qtimetamux: qtimetamux: Meta muxer qtimetatransform: qtimetatransform: Meta Transform qtimlaclassification: qtimlaclassification: Machine Learning audio classification qtimlaconverter: qtimlaconverter: Machine Learning Audio Converter qtimldemux: qtimldemux: Batching stream buffers qtimlmetaextractor: qtimlmetaextractor: Video mlmeta extractor qtimlmetaparser: qtimlmetaparser: Meta parser qtimlpostprocess: qtimlpostprocess: Machine Learning postprocess qtimlqnn: qtimlqnn: QNN based ML plugin qtimlsnpe: qtimlsnpe: SNPE Machine Learning qtimltflite: qtimltflite: TFLite Machine Learning qtimlvclassification: qtimlvclassification: Machine Learning image classification qtimlvconverter: qtimlvconverter: Machine Learning Video Converter qtimlvdetection: qtimlvdetection: Machine Learning image object detection qtimlvpose: qtimlvpose: Machine Learning Pose qtimlvsegmentation: qtimlvsegmentation: Machine Learning image segmentation qtimlvsuperresolution: qtimlvsuperresolution: Machine Learning image super resolution qtimsgpub: qtimsgpub: Message Publisher Client qtimsgsub: qtimsgsub: Message Subscriber Client qtiobjtracker: qtiobjtracker: Object Tracker qtiqmmfsrc: qtiqmmfsrc: QMMF Video Source qtiredissink: qtiredissink: QTI Redis Sink Element qtirestrictedzonedbg: qtirestrictedzonedbg: Restricted Zone Filter qtirtspbin: qtirtspbin: RTSP streaming Bin qtismartvencbin: qtismartvencbin: Smart Video Encode Bin qtisocketsink: qtisocketsink: QTI Socket Sink Element qtisocketsrc: qtisocketsrc: QTI Socket Source Element qtiuridecodebin: qtiuridecodebin: Generic bin qtivcomposer: qtivcomposer: Video composer qtivideotemplate: qtivideotemplate: Video template qtivoverlay: qtivoverlay: Video Overlay qtivsplit: qtivsplit: Video stream splitter qtivtransform: qtivtransform: Video transformer typefindfunctions: image/x-quicktime: qif, qtif, qti
Typical pipeline patterns
A practical way to think about RUBIK Pi 3 pipelines is to group them by intent:
| Intent | Typical stages |
|---|---|
| Preview | camera → colorspace / memory handling → display |
| Snapshot | camera → conversion → JPEG encode → file sink |
| Record | camera → transform / overlay → encoder → mux → file sink |
| AI preview | camera → resize / preprocess → inference → overlay → display |
| Network stream | camera → transform / encode → RTP / RTSP / WebRTC sink |
Camera capture pipelines
Camera capture smoke test
The simplest camera capture pipeline is:
gst-launch-1.0 qtiqmmfsrc ! fakesink silent=false -v
If the capture succeeds, the fakesink will print one message in the command line for each frame captured. Following is an example of the command line output expected:
Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)qmmfsrc/video_0, f0 /GstPipeline:pipeline0/GstQmmfSrc:qmmfsrc0.GstQmmfSrcVideoPad:video_0: caps = image/jpeg, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"image/jpeg\,\ width\=\(int\)640\,\ h0 /GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = image/jpeg, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1 Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"segment, flags=(GstSegme0 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll ******* Redistribute latency... /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (42289 bytes, dts: none, pts: 0:00:00.337249179, duration: 0:00:00.033333333, offset: -10 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (42551 bytes, dts: none, pts: 0:00:00.369826001, duration: 0:00:00.033333333, offset: -10 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (42285 bytes, dts: none, pts: 0:00:00.402402824, duration: 0:00:00.033333333, offset: -10
Camera Capture to display
The following pipeline requires the HDMI display to be connected:
gst-launch-1.0 qtiqmmfsrc ! waylandsink
The expected result is a window pop-up on the display showing the live camera capture.
Validated capture example
The current official camera documentation provides a straightforward image-capture example using `qtiqmmfsrc` and `jpegenc`.
gst-launch-1.0 -e qtiqmmfsrc camera=0 ! \ video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! \ queue ! jpegenc ! queue ! \ multifilesink location=/opt/img0_%d.jpg max-files=5
This is a good first validation because it proves several things at once:
- the sensor is detected,
- the CSI link is working,
- the software stack can produce frames,
- and the pipeline is stable enough to write files.
Preview pipeline pattern
A simple live-preview pipeline usually follows this structure:
qtiqmmfsrc ! capsfilter ! queue ! sink
Template:Fact check required Validate and publish one exact preview pipeline on the current Ubuntu image before adding it as a canonical copy-paste command. Display sinks, memory caps, and zero-copy behavior may differ across image releases.
Transform, resize, and zero-copy considerations
In AI and multimedia products, the expensive part is often not inference itself but everything around it: resize, color conversion, crop, frame copies, synchronization, and rendering. Current Qdemo documentation highlights `qtivtransform` as the component used for accelerated transform work such as color conversion, cropping, and resizing.
That is important because well-designed transform stages can reduce CPU load, improve frame stability, and make the downstream AI runtime easier to feed.
Template:Add benchmark data Add measured CPU, memory, and FPS comparisons between software-only transforms and accelerated transforms on a fixed image release and camera resolution.
AI-assisted multimedia pipelines
The Ubuntu Qdemo documentation describes a software path where capture, transform, inference, and rendering are orchestrated through GStreamer-oriented components. This is the right mental model for RUBIK Pi 3 AI development: the AI model is one stage in a multimedia graph, not a separate world.
Example conceptual flow:
Camera (qtiqmmfsrc)
↓
Transform / resize (qtivtransform)
↓
Inference runtime
↓
Overlay / metadata rendering
↓
Display, recording, or streaming output
See also Rubik Pi 3/AI and Computer Vision.
Recording and streaming
Recording
A robust recording pipeline typically adds the following stages beyond preview:
camera → transform → encoder → parser → muxer → filesink
Whether you use software or hardware-backed encode paths depends on the image and plugin availability. The right choice should be validated against your specific image version, resolution, latency target, and storage medium.
Streaming
For products that need browser, server, or device-to-device delivery, RidgeRun commonly builds on GStreamer to expose RTSP, RTP, and WebRTC paths. Those same design patterns translate well to QCS6490-class platforms.
Useful RidgeRun references:
- Introduction to RidgeRun's GstWebRTC
- RidgeRun Multimedia Streaming Solutions: RTSP, WebRTC, RTP, and ONVIF Integration Tools
RidgeRun GStreamer technologies relevant to RUBIK Pi 3
GstInference
GstInference is RidgeRun's framework for embedding deep learning inference directly into GStreamer pipelines. Even when a project starts from vendor sample applications, GstInference is useful as an architectural reference for building maintainable inference-enabled media graphs.
GstWebRTC
Introduction to RidgeRun's GstWebRTC and related streaming documentation are relevant when the product needs live browser delivery, operator interfaces, or remote preview.
Camera-driver and sensor integration
GStreamer pipelines are only as reliable as the underlying capture path. If the project depends on a non-reference sensor or custom module, combine this page with RidgeRun Linux Camera Drivers and Mira220 Camera Driver for Rubik Pi 3.
Debugging GStreamer on RUBIK Pi 3
When a pipeline fails, debug in this order:
- verify board power and image version,
- verify the camera module and cable orientation,
- validate capture with the simplest possible pipeline,
- confirm caps and frame size assumptions,
- add transforms one stage at a time,
- then add inference, encode, or network sinks.
Good debug habits include:
- logging `/etc/os-release`,
- recording exact pipeline strings,
- capturing kernel / journal messages,
- and testing with one known-good camera module before introducing custom hardware.
For general help, see GStreamer.
Common issues
Could not initialise Wayland output
The pipeline fails with:
0:00:00.066303490 1824 0x55868e9e00 WARN waylandsink gstwaylandsink.c:385:gst_wayland_sink_find_display:<waylandsink0> warning: Could not initialise Wayland output 0:00:00.066341250 1824 0x55868e9e00 WARN waylandsink gstwaylandsink.c:385:gst_wayland_sink_find_display:<waylandsink0> warning: Failed to create GstWlDisplay: 'Failed to connect to the wayland display '(default)''
Solution: Export the display environment variables:
export XDG_RUNTIME_DIR=/dev/socket/weston export WAYLAND_DISPLAY=wayland-1
Failed to Open Camera
The pipeline fails with:
Setting pipeline to PAUSED ... 0:00:00.083557969 1868 0x558a260e50 ERROR qtiqmmfsrc qmmf_source_context.cc:1458:gst_qmmf_context_open: QMMF Recorder StartCamera Failed! 0:00:00.083686823 1868 0x558a260e50 WARN qtiqmmfsrc qmmf_source.c:1217:qmmfsrc_change_state:<qmmfsrc0> error: Failed to Open Camera! ERROR: from element /GstPipeline:pipeline0/GstQmmfSrc:qmmfsrc0: Failed to Open Camera! Additional debug info: /usr/src/debug/qcom-gstreamer1.0-plugins-oss-qmmfsrc/1.0/qmmf_source.c(1217): qmmfsrc_change_state (): /GstPipeline:pipeline0/GstQmmfSrc:qmmfsrc0 ERROR: pipeline doesn't want to preroll. Failed to set pipeline to PAUSED. Setting pipeline to NULL ... Freeing pipeline ...
Solution: There is not a single root cause, but these questions might help resolve the problem:
- Is the camera connected correctly?
Refer to the image below.
- Does the /var/cache/camera/camxoverridesettings.txt has the following contents?
root@rubikpi:~# cat /var/cache/camera/camxoverridesettings.txt multiCameraLogicalXMLFile=kodiak_dc.xml enableNCSService=FALSE
If the file does not exist, or has different contents, you can overwrite it as follows:
echo multiCameraLogicalXMLFile=kodiak_dc.xml > /var/cache/camera/camxoverridesettings.txt echo enableNCSService=FALSE >> /var/cache/camera/camxoverridesettings.txt systemctl restart cam-server.service
Then try again.
Suggested development workflow
1. Validate camera with qtiqmmfsrc + JPEG capture 2. Add live preview 3. Add transform / resize 4. Add inference or encode path 5. Add metadata overlay 6. Add network streaming or ROS 2 bridge 7. Measure CPU, FPS, latency, and memory
This staged approach avoids the common trap of debugging six variables at once.
Frequently asked questions
- Can RUBIK Pi 3 run GStreamer pipelines?
- Yes. Current official Ubuntu material includes camera and AI workflows built around GStreamer-oriented components, and the platform is well-suited to preview, record, inference, and streaming graphs.
- What source element is used for cameras?
- Current official camera documentation uses `qtiqmmfsrc` for CSI camera capture on the board.
- How do I test the camera quickly with GStreamer?
- Use a simple `qtiqmmfsrc` pipeline that captures a few JPEG frames to disk before attempting more complex preview or inference graphs.
- Can I combine GStreamer with AI on RUBIK Pi 3?
- Yes. The vendor Ubuntu stack and RidgeRun technologies both support the idea of inference as one stage inside a larger multimedia pipeline.
- Where does RidgeRun help most with GStreamer on this platform?
- RidgeRun typically helps with stable capture, zero-copy design, AI integration, overlays, low-latency streaming, and product-grade pipeline architecture.
Related pages
- Rubik Pi 3
- Rubik Pi 3/Hardware Overview
- Rubik Pi 3/Software Overview
- Rubik Pi 3/AI and Computer Vision
- GStreamer
- GstInference
- Introduction to RidgeRun's GstWebRTC
=
GstQtOverlay
export XDG_RUNTIME_DIR=/dev/socket/weston export WAYLAND_DISPLAY=wayland-1 export QT_QPA_PLATFORM=wayland export QT_QPA_PLATFORM=wayland-egl also works. gst-launch-1.0 videotestsrc is-live=1 ! "video/x-raw,width=1280,height=720,format=RGBA" ! qtoverlay qml=main.qml ! queue ! videoconvert ! glimagesink or gst-launch-1.0 -e qtiqmmfsrc camera=0 antibanding=0 ! 'video/x-raw(memory:GBM),format=NV12,width=1280,height=720,framerate=30/1' ! qtivtransform ! qtoverlay qml=main_drone.qml ! glimagesink or gst-launch-1.0 -e qtiqmmfsrc camera=0 antibanding=0 ! 'video/x-raw(memory:GBM),format=NV12,width=1280,height=720,framerate=30/1' ! qtivtransform ! qtoverlay qml=main_drone.qml ! qtivtransform ! waylandsink fullscreen=true async=true