RidgeRun Media Microservice
RidgeRun's Media Service connects to several RTSP streams and performs the following operations:
- People detection.
- Re-identification (ReID) to maintain the person ID across multiple cameras.
- Generates a Birds Eye View (BEV) video of the combined video streams
- Calculates the direction in which each person is looking
- Posts each detection with position and direction as a redis message.
- Generates an RTSP stream with the BEV video.
The following picture shows a basic diagram of the service
The service can be configured using a configuration file or via REST API.
Service Configuration
The service can be configured in 2 different ways: providing a configuration file via the --config_file CLI argument or via the /configuration request (see the [service API documentation]). In both cases, the configuration will be in JSON format as follows:
{
"inputs": [
{
"id": "camera0",
"index": 1
},
{
"id": "camera1",
"index": 0
}
],
"output": {
"port": 6062,
"resolution": {
"width": 1920,
"height": 1080
},
"mapping": "bev",
"bitrate": 4000000,
"brightness": 1.0
},
"cam_position": {
"x": 1166,
"y": 900
},
"head_pose_confidence": 0.1
}
The configuration entries are as follows:
- inputs: This is the list of cameras it will use for detections and for generating the BEV.
- id: The camera id as specified in the Camera Service.
- index: The camera index that will be used to position the camera in the final BEV.
- output: BEV output stream configuration.
- port: RTSP port to be used for the output stream.
- resolution: The output stream resolution
- width: Output width
- height Output height
- mapping: RTSP mapping to be used.
- bitrate: Output bitrate (the output stream is H264 encoded).
- brightness: Output brightness. Used only if the PVA brightness element is available
- cam_position: This is the camera position in the BEV output. It will be used to determine head direction.
- x: Camera X coordinate
- y: Camera Y coordinate
- head_pose_confidence: The minimum confidence to consider a head pose (direction) estimation as valid.
In addition to that, a BEV calibration file has to be given using the --bev_calibration_file argument. The indexes in this calibration file have to match the ones in the configuration file. See the BEV Calibration Guide for instruction on how to generate a calibration file.
People Detection and Head Direction Estimation
The Media Service performs people detection and head direction estimation, which can later be used to determine people's engagement, among other things. The person position and head direction are reported via Redis messages using rrms-utils DirectionSchemaGenerator which has the following format.
{
"id": 1,
"cameraid": "cam1",
"timestamp": "2025-01-01T00:00:00Z",
"width": 1920,
"height": 1080,
"detections": [
{
"objectid": "object1",
"position": {
"x": 10,
"y": 20
},
"direction": {
"x": 1,
"y": 0
}
}
]
}
The detections field is an array with the object ID, its position and its direction vector.
The position and direction coordinates are reported referenced to the BEV output video and for the direction estimation, the camera position provided in configuration is used.
PVA Brightness Support
The service can optionally offer a brightness functionality to the BEV output via PVA. This functionality will be automatically added if the pvabrightness element is found on the system. If available, it can be controlled either via the /configuration or the dedicated /brightness request (see [URL API documentation] for details).
Running the Service
Using Docker
You can obtain or build a docker image for the media Microservice, check below the method of your preference. The image includes a base L4T image and the dependencies to run the Media Microservice application. The image was developed and tested for NVIDIA JetPack 6.0. Once you get your image with either method proceed to Launch the container
Pre-build Image
Once you have obtained the media-service docker image, you can install it in your jetson with the following command:
docker load < media-service.tar.gz
Build Image
You can build the media service image using the Dockerfile in the docker directory. First, we need to prepare the context directory for this build. You need to create a directory and include this repository and the remaining dependencies. The Dockerfile will look for all the packages in the context directory and copy them to the container.
After this, your context directory should look like this:
media-context/ ├── gst-pva-brightness ├── gst-rr-track-id ├── gst-rtsp-sink ├── libpanorama ├── media ├── pva-sdk-and-algorithm-files └── rrms-utils
Remember that gst-pva-brightness and pva-sdk-and-algorithm-files are optional.
Then, build the container image with the following command:
docker build \ --network=host \ -f Dockerfile \ -t ridgerun/media-service:latest \ context/
Change context/ to your context's path and the tag to the name you want to give to the image.
If you want the PVA feature, you need to set --target pva flag at the image build as follows:
docker build \ --network=host \ --target pva \ -f Dockerfile \ -t ridgerun/media-service:latest \ context/
Launch the container
You can ensure the image has been created successfully by running
docker images
You should get an entry showing the ridgerun/media-service image
nvidia@ubuntu:~$ docker images REPOSITORY TAG ridgerun/media-service latest
The container can be launched by running the following command:
docker run -it \ --network host \ --volume <path to directory containing your configuration file>:/configs \ --name media-service \ ridgerun/media-service:latest \ --config-file /configs/<your configuration file name>
Here we are creating a container called media-service that will start the media service application (check Using Standalone Application), launching the server in 127.0.0.1 and port 5051. When the service starts, it will communicate with both camera and redis , so make sure both services are running.
After starting the service, you should see an entry with the media-service container:
nvidia@ubuntu:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e98d03ae7d2e ridgerun/media-service "media --config_file…" 41 hours ago Up 6 hours media-service
Using Standalone Application
The media service can also be used outside of the docker container. You can install it by running the following command from the project directory:
pip install .
Then you will have the service with the following options:
media --help
usage: media [-h] [--config_file CONFIG_FILE] [--port PORT] [--host HOST] [--bev_calibration_file BEV_CALIBRATION_FILE]
options:
-h, --help show this help message and exit
--config_file CONFIG_FILE
JSON file with media initial configuration
--port PORT Port for server
--host HOST Server ip address
--bev_calibration_file BEV_CALIBRATION_FILE
BEV calibration file
To start the service in address 127.0.0.1 and port 5053 just run:
media
If you want to serve in a different port or address, use the --port and --host options.
If you want to start with a configuration, you can use the --config-file argument to specify the path to your configuration.