Qualcomm Robotics RB5/Demos/Drone Demo

From RidgeRun Developer Wiki

Introduction

This demo consists of a simulated drone system on a Qualcomm Robotics RB5 that performs live capture, image processing, sensor readings, metadata, encoding and network transmission. Some of the key components it explores include:

  • PTZR for dynamic control of the imager
  • Accelerometer readings embedded as metadata in the stream
  • H265 encoding
  • RTSP streaming
  • A media server for media management


The following image summarizes the overall functionality.


Dependencies

In order to accomplish the desired behavior, we use some of our products with out-of-the-box functionalities that significantly reduce development time. For this demo, we will be using:

Run the demo

Once you get the code, you should see the following files: 

├── requirements.txt
├── drone_demo.py 
├── rrdronedemo
│   ├── __init__.py
│   ├── argumenthandler.py
│   ├── imu.py
│   ├── pipelineentity.py
│   └── tcp_server.py

The demo can be easily run with the following command: 

python3 drone_demo.py

Application parameters

For a flexible configuration, we exposed the following arguments which can be used to tune the application. This includes:

Option Type Description
--rotate_level float Rotation (in degrees) about the depth axis with origin on the center of the capture region in the range [-180.0, 180.0]
--tilt_level float Translation about the y axis starting from the center of the input image in the range [-image height, image height]
--pan_level float Translation about the x axis starting from the center of the input image in the range [-image width, image width]
--zoom_level float Zoom level
--resolution string Camera input resolution {720p,1080p,4k}
--control_rate int Bitrate control method: (0) Disable (1) Constant (2) CBR-VFR (3) VBR-CFR (4) VBR-VFR (5) CQ
--bitrate unsigned int Target bitrate in bits per second (0 is no specific bitrate)
--idr_interval unsigned int IDR frame interval (0 means no specific IDR)
--rtsp-port unsigned int RTSP port
--mapping string RTSP mappings
--auth string Authentication and authorization. Format: user1:password1
--tcp-port unsigned int TCP server port for receiving commands

Control through TCP

This demo allows the user to control the media server through a TCP connection, which can be established using the following command: 

nc <IP_ADDRESS> <TCP_PORT>

The commands supported include:

For PTZR control:

  • zoom
  • rotate
  • tilt
  • pan_level

For the media server:

  • play
  • pause
  • stop

Sample usage: 

nc 192.168.23.180 9999
zoom=5
rotate=1
tilt=3
pan_level=4
play
pause
stop


Capture metadata

Finally, if you want to inspect the received metadata, you can use the seiextract element as follows: 

GST_DEBUG=*seiextract*:MEMDUMP gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:5000/stream1 ! rtph265depay ! video/x-h265 ! seiextract !  h265parse ! avdec_h265 ! videoconvert ! autovideosink
Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Qualcomm_Robotics_RB5/Demos/Drone_Demo&oldid=53593"