View source for Metropolis Microservices/RidgeRun Services/Media

<noinclude>
{{Metropolis Microservices/Head
|previous=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Services/Engagement Analytics
|next=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Microservices Demo
|title=RidgeRun Media Microservice
|description=This page provides a general overview of the Media RidgeRun Service
}}
</noinclude>

RidgeRun's Media Service connects to several RTSP streams and performs the following operations:

* People detection.
* Re-identification (ReID) to maintain the person ID across multiple cameras.
* Generates a Birds Eye View (BEV) video of the combined video streams
* Calculates the direction in which each person is looking
* Posts each detection with position and direction as a redis message.
* Generates an RTSP stream with the BEV video.

The following picture shows a basic diagram of the service

[[File:MicroservicesArchitecture-media.png|800px|thumb|center]]

The service can be configured using a configuration file or via REST API.

{{message|title='''TL;DR'''
  |content=
You can run this microservice with the following command:

<syntaxhighlight lang="bash">
docker run \
-it \
--network host \
--name media-service \
ridgerun/media-service:latest
</syntaxhighlight>

This will run the Media service in localhost using port 5051. 

'''NOTE:''' The first time the service runs it will take about 1h to generate the engine it needs to perform people detection.
}}

{{message|type=doc
  |content=
* [https://ridgerun.github.io/media-service/Service Documentation]
* [https://ridgerun.github.io/media-service/api.html Api Documentation]
<br>
See [[Metropolis Microservices/Tools/API Usage|API Usage]] for an example of how to use the API documentation to control the service
}}

== Service Configuration ==

The service can be configured in 2 different ways: providing a configuration file via the '''--config_file''' CLI argument or via the '''/configuration''' request (see the [service API documentation]). In both cases, the configuration will be in JSON format as follows:

<syntaxhighlight lang="JSON">
{
    "inputs": [
        {
            "id": "camera0",
            "index": 1
        },
        {
            "id": "camera1",
            "index": 0
        }
    ],
    "output": {
        "port": 6062,
        "resolution": {
            "width": 1920,
            "height": 1080
        },
        "mapping": "bev",
        "bitrate": 4000000,
	"brightness": 1.0
    },
    "cam_position": {
        "x": 1166,
        "y": 900
    },
    "head_pose_confidence": 0.1
}
</syntaxhighlight>

The configuration entries are as follows:

*'''inputs:''' This is the list of cameras it will use for detections and for generating the BEV.
**'''id:''' The camera id as specified in the [https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun_Services/Camera Camera Service].
**'''index:''' The camera index that will be used to position the camera in the final BEV.
*'''output:''' BEV output stream configuration.
**'''port:''' RTSP port to be used for the output stream.
**'''resolution:''' The output stream resolution
***'''width:''' Output width
***'''height''' Output height
**'''mapping:''' RTSP mapping to be used.
**'''bitrate:''' Output bitrate (the output stream is H264 encoded).
**'''brightness:''' Output brightness. '''Used only if the PVA brightness element is available'''
*'''cam_position:''' This is the camera position in the BEV output. It will be used to determine head direction.
**'''x:''' Camera X coordinate
**'''y:''' Camera Y coordinate
*'''head_pose_confidence:''' The minimum confidence to consider a head pose (direction) estimation as valid.

In addition to that, a BEV calibration file has to be given using the '''--bev_calibration_file''' argument. The indexes in this calibration file have to match the ones in the configuration file. See the [https://developer.ridgerun.com/wiki/index.php/Birds_Eye_View/Calibration_Guide#Understanding_the_Calibration_Process BEV Calibration Guide] for instruction on how to generate a calibration file.

== People Detection and Head Direction Estimation == 

The Media Service performs people detection and head direction estimation, which can later be used to determine people's engagement, among other things. 
The person position and head direction are reported via Redis messages using [https://gitlab.ridgerun.com/ridgerun/rnd/jetson-microservices-and-genai/rrms-utils rrms-utils] '''DirectionSchemaGenerator''' which has the following format. 

<syntaxhighlight lang=JSON>
{
    "id": 1,
    "cameraid": "cam1",
    "timestamp": "2025-01-01T00:00:00Z",
    "width": 1920,
    "height": 1080,
    "detections": [
        {
            "objectid": "object1",
            "position": {
                "x": 10,
                "y": 20
            },
            "direction": {
                "x": 1,
                "y": 0
            }
        }
    ]
}
</syntaxhighlight>

The detections field is an array with the object ID, its position and its direction vector.

The position and direction coordinates are reported referenced to the BEV output video and for the direction estimation, the camera position provided in configuration is used.

== PVA Brightness Support ==

The service can optionally offer a brightness functionality to the BEV output via PVA. This functionality will be automatically added if the pvabrightness element is found on the system. If available, it can be controlled either via the '''/configuration''' or the dedicated '''/brightness''' request (see [URL API documentation] for details).

{{message| Check [https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun_Services/Media#Using_Docker Using Docker] section for instructions on how to enable PVA in the service docker container.}}

== Running the Service ==

=== Using Docker ===

You can obtain or build a docker image for the media Microservice, check below the method of your preference. The image includes a base L4T image and the dependencies to run the Media Microservice application. The image was developed and tested for NVIDIA JetPack 6.0. Once you get your image with either method proceed to [[Metropolis_Microservices/RidgeRun_Services/Media#Launch_the_container|Launch the container]]

==== Pre-build Image ====

{{message|[https://www.ridgerun.com/contact Contact Ridgerun] if you want an evaluation version of the media service docker image.}}

Once you have obtained the media-service docker image, you can install it in your jetson with the following command:
 
<syntaxhighlight lang="bash">
docker load < media-service.tar.gz
</syntaxhighlight>

==== Build Image ====

{{message|[https://www.ridgerun.com/contact Contact Ridgerun] if you want access to the media service source code.}}

You can build the media service image using the Dockerfile in the docker directory.  First, we need to prepare the context directory for this build. You need to create a directory and include this repository and the remaining dependencies. The Dockerfile will look for all the packages in the context directory and copy them to the container.

After this, your context directory should look like this:

<syntaxhighlight lang="bash">
media-context/
├── gst-pva-brightness
├── gst-rr-track-id
├── gst-rtsp-sink
├── libpanorama
├── media
├── pva-sdk-and-algorithm-files
└── rrms-utils

</syntaxhighlight>Remember that gst-pva-brightness and pva-sdk-and-algorithm-files are optional.{{message
|content=
You can obtain rrms-utils with:
<pre>
git clone https://gitlab.ridgerun.com/open/ridgerun-microservices/rrms-utils.git
</pre>
}}

Then, build the container image with the following command:

<syntaxhighlight lang="bash">
docker build \
--network=host \
-f Dockerfile \
-t ridgerun/media-service:latest \
context/

</syntaxhighlight>

Change context/ to your context's path and the tag to the name you want to give to the image.

If you want the PVA feature, you need to set '''--target pva''' flag at the image build as follows:

<syntaxhighlight lang="bash">
docker build \
--network=host \
--target pva \
-f Dockerfile \
-t ridgerun/media-service:latest \
context/


</syntaxhighlight>

====  Launch the container ==== 

You can ensure the image has been created successfully by running

<syntaxhighlight lang="bash">
docker images
</syntaxhighlight>

You should get an entry showing the ridgerun/media-service image
<syntaxhighlight lang="bash">
nvidia@ubuntu:~$ docker images
REPOSITORY                                  TAG
ridgerun/media-service                    latest
</syntaxhighlight>

The container can be launched by running the following command:

<syntaxhighlight lang="bash">
docker run -it \
--network host \
--volume <path to directory containing your configuration file>:/configs \
--name media-service \
ridgerun/media-service:latest  \
--config-file /configs/<your configuration file name>

</syntaxhighlight>


Here we are creating a container called media-service that will start the media service application (check [[Metropolis Microservices/RidgeRun Services/Media#Using Standalone Application|Using Standalone Application]]), launching the server in 127.0.0.1 and port 5051. When the service starts, it will communicate with both camera and redis , so make sure both services are running.  

After starting the service, you should see an entry with the media-service container:
<syntaxhighlight lang="bash">
nvidia@ubuntu:~$ docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED          STATUS         PORTS     NAMES
e98d03ae7d2e   ridgerun/media-service          "media --config_file…"   41 hours ago   Up 6 hours                                                  media-service

</syntaxhighlight>

=== Using Standalone Application ===

The media service can also be used outside of the docker container. You can install it by running the following command from the project directory:

<syntaxhighlight lang="bash">
pip install .
</syntaxhighlight>

Then you will have the service with the following options:

<syntaxhighlight lang="bash">
media --help
usage: media [-h] [--config_file CONFIG_FILE] [--port PORT] [--host HOST] [--bev_calibration_file BEV_CALIBRATION_FILE]

options:
  -h, --help            show this help message and exit
  --config_file CONFIG_FILE
                        JSON file with media initial configuration
  --port PORT           Port for server
  --host HOST           Server ip address
  --bev_calibration_file BEV_CALIBRATION_FILE
                        BEV calibration file
</syntaxhighlight>

To start the service in address 127.0.0.1 and port 5053 just run:
<syntaxhighlight lang="bash">
media
</syntaxhighlight>

If you want to serve in a different port or address, use the '''--port''' and '''--host''' options.

If you want to start with a configuration, you can use the '''--config-file''' argument to specify the path to your configuration.

<noinclude>
{{Metropolis Microservices/Foot
|previous=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Services/Engagement Analytics
|next=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Microservices Demo
}}
</noinclude>