Qualcomm Robotics RB5 - Demos - Drone Demo

From RidgeRun Developer Wiki



⇐ Index ⇒


Qualcomm Robotics RB5/RB6
Introduction
SoM Overview
Why the RB5/RB6?
Carrier Boards
Software Development Support
Boot Devices
Thundercomm SDK
Linaro
Special / Debug Modes
Development in the Board
Getting into the Board
Device Setup
Build from source
Base Image
Customize Image
Capture Subsystem
Hardware Capture Components
Software Capture Components
Building your own camera sensor driver
Image Processing and Acceleration Software
GPU Profiling
Fast CV
GStreamer Pipelines
Capture and Display
Video Encoding
Video Decoding
AI Acceleration
Neural Processing SDK
TensorFlow
Sensors
IMU
Demos
Contact Us




Introduction

This demo consists of a simulated drone system on a Qualcomm Robotics RB5 that performs live capture, image processing, sensor readings, metadata, encoding and network transmission. Some of the key components it explores include:

  • PTZR for dynamic control of the imager
  • Accelerometer readings embedded as metadata in the stream
  • H265 encoding
  • RTSP streaming
  • A media server for media management

The following image summaries the overall functionality.



Load video
YouTube

Interested in the Full Demo?

Send us a message through the Contact Us form with the following details:

  • Platform: i.e. Qualcomm RB5 Devkit, Lantronix, Thundercomm
  • OS: i.e. LU Ubuntu 20.04
  • Brief Description of your use case

Important: Our demo is available only for the RB5 with Ubuntu 20.04

Dependencies

In order to accomplish the desired behaviour, we use some of our products with out-of-the-box functionalities that significantly reduce development time. For this demo, we will be using:

All these products are contained in the demo in their evaluation versions. If a professional version is required, please, Contact Us.

Installation

Please, install all the dependencies using: 

bash pre-install.sh

After that, use the meson build system and install the demos and dependencies: 

meson builddir --prefix=/usr
ninja -C builddir
ninja -C builddir install

Run the demo

Once you get the code, you should see the following files: 

├── requirements.txt
├── drone_demo.py 
├── rrdronedemo
│   ├── __init__.py
│   ├── argumenthandler.py
│   ├── imu.py
│   ├── pipelineentity.py
│   └── tcp_server.py

Environment

Please, enable the HDMI with the following instructions: Qualcomm_Robotics_RB5/Development_in_the_Board/Device_Setup/HDMI_Setup and set the following environment variables (if you are remotely connected to the RB5): 

export WAYLAND_DISPLAY=wayland-0
export XDG_RUNTIME_DIR=/usr/bin/weston_socket

Running

The demo can be easily run with the following command: 

python3 drone_demo.py

Application parameters

For a flexible configuration, we exposed the following arguments which can be used to tune the application. This includes:

Option Type Description
--rotate_level float Rotation (in degrees) about the depth axis with origin on the center of the capture region in the range [-180.0, 180.0]
--tilt_level float Translation about the y axis starting from the center of the input image in the range [-image height, image height]
--pan_level float Translation about the x axis starting from the center of the input image in the range [-image width, image width]
--zoom_level float Zoom level
--resolution string Camera input resolution {720p,1080p,4k}
--control_rate int Bitrate control method: (0) Disable (1) Constant (2) CBR-VFR (3) VBR-CFR (4) VBR-VFR (5) CQ
--bitrate unsigned int Target bitrate in bits per second (0 is no specific bitrate)
--idr_interval unsigned int IDR frame interval (0 means no specific IDR)
--rtsp-port unsigned int RTSP port
--mapping string RTSP mappings
--auth string Authentication and authorization. Format: user1:password1
--tcp-port unsigned int TCP server port for receiving commands


Control through TCP

This demo allows the user to control the media server through a TCP connection, which can be established using the following command: 

nc <IP_ADDRESS> <TCP_PORT>

Please, execute this command from your computer or another machine. The IP_ADDRESS is the code of the RB5 and the default TCP_PORT is 9999 by default. It can by changed through command arguments when executing the demo.


The supported commands include:

For PTZR control:

  • zoom
  • rotate
  • tilt
  • pan

For the media server:

  • play
  • pause
  • stop

Sample usage: 

nc 192.168.23.180 9999
zoom=5
rotate=1
tilt=3
pan=4
play
pause
stop

Capture metadata

Finally, if you want to inspect the received metadata, you can use the seiextract element as follows: 

GST_DEBUG=*seiextract*:MEMDUMP gst-launch-1.0 rtspsrc location=rtsp://$IP_ADDRESS:49152/stream1 ! rtph265depay ! video/x-h265 ! seiextract !  h265parse ! avdec_h265 ! videoconvert ! autovideosink

This code is executed on another machine with a display attached to it and connected to the same network as the RB5. If you don't have the seiextract, you can execute the following: 

gst-launch-1.0 rtspsrc location=rtsp://$IP_ADDRESS:49152/stream1 ! rtph265depay ! video/x-h265 !  h265parse ! avdec_h265 ! videoconvert ! autovideosink

Please, note that the RTSP port is by default 49152 and the IP_ADDRESS must contain the IP address of the RB5

Looks good, right?

Are you interested in getting through this demo and more solutions like this? Feel free to Contact Us.

Known Issues

1. The demo shows a segfault when running it

This is due to a misconfiguration on the display. Make sure of connecting a display, enabling the HDMI and setting the environment variables.

2. There is an error associated to GLX context

This is due to a misconfiguration in the display. Please, send stop and play again.


⇐ Index ⇒



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Qualcomm_Robotics_RB5/Demos/Drone_Demo&oldid=54132"