Metropolis Microservices/RidgeRun Microservices Demo/Demo Description: Difference between revisions

Latest revision as of 19:44, 3 September 2024

‎

Metropolis Microservices for NVIDIA Jetson

Table of Contents [Sticky]

The SmartSeek360 searches for objects specified by the user in a 360-degree video stream, once the object is found the system follows it through the video leveraging the PTZ functions, and also triggers a video recording.

In the following diagram; Blocks in green are Microservices designed by NVIDIA and blocks in blue are custom Microservices developed by RidgeRun.

This system is made up of the following Microservices:

NVIDIA Metropolis Microservices:

Video Storage Toolkit (VST): VST NVIDIA microservice auto-discovers ONVIF-S compliant IP cameras, and allows the use of custom IP stream as video source. It then allows for video to be stored, played back at various speeds, or paused at any frame
REDIS: Redis is a message broker that contains all device metadata. VST depends on Redis streams to publish device status change events. DeepStream depends on Redis streams to publish metadata and receive VST camera status events.
INGRESS: Ingress is the API Gateway to all Metropolis Microservices APIs. It acts as a reverse proxy, where incoming API requests pass through it, which it then forwards to the appropriate microservice based on path prefix.

RidgeRun Microservices:

PTZ Microservice: The PTZ Microservice is designed to receive 360-degree equirectangular RTSP video streams and outputs an RTSP stream with the performed PTZ operations over the equirectangular videos using RidgeRun libpanorama.
Detection Microservice: Detection Microservice detects in the input stream the target objects described in a text prompt.The microservice uses the NanoOwl generative AI model that allows open vocabulary detection. Meaning that the user can provide a list of objects that are not bound to any specific classes. For example, it is possible to indicate: search for dogs.
AI Agent: AI Agent Microservice allows a natural communication between the user and other microservices. This service uses the Hugging Face LLM Trelis/Llama-2-7b-chat-hf-function-calling-v3 to convert text commands into API calls, process the LLM result, and call the corresponding API request. For example, it is possible to request: move the camera 30 degrees to the Right.
Analytics: This Microservice reads detection metadata provided by the REDIS NVIDIA microservice and moves the camera to the detected object position using PTZ and starts video recordings using the NVIDIA VST microservice.

@@ Line 1: / Line 1: @@
 <noinclude>
-{{Metropolis Microservices/Head|previous=Getting_Started/Introduction|next=Application_services|metakeywords=Metropolis|title= Demo Description |metadescription=This page provides a general overview of the PTZ RidgeRun Service}}
+{{Metropolis Microservices/Head
+|previous=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Microservices Demo
+|next=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Microservices Demo/Getting the code
+|metakeywords=Metropolis
+|title= Demo Description
+|metadescription=This page provides a general overview of the PTZ RidgeRun Service
+}}
 </noinclude>
-The '''SmartSeek360 ''' search for objects specified by the user in a 360-degree video stream, once the object is found the system follows it trough the video leveraging the PTZ functions, and triggers video recording at the same time.
+The '''SmartSeek360 ''' searches for objects specified by the user in a 360-degree video stream, once the object is found the system follows it through the video leveraging the PTZ functions, and also triggers a video recording.
-{{Review|We should add a diagram of the demo}}
+In the following diagram; Blocks in green are Microservices designed by NVIDIA and blocks in blue are custom Microservices developed by RidgeRun.
-This system is made up of the following Microservices:
-'''NVIDIA Metropolis Microservices:'''
+[[File:RidgeRun_Microservices_Demo_diagram-vertical.png|thumbnail|center|640px|center|'''RidgeRun SmartSeek360''' - Microservices Demo diagram]]
-#'''Video Storage Toolkit (VST):''' VST NVIDIA microservice auto-discovers ONVIF-S compliant IP cameras, and allows use of custom IP stream as video source. It then allows for video to be stored, played back at various speeds, or paused at any frame
-#'''REDIS: ''' Redis is a message broker that contains all device metadata. VST depends on Redis streams to publish device status change events. DeepStream depends on Redis streams to publish metadata and receive VST camera status events.
-#'''INGRESS: '''
-{{Review|Add links to the respective MS nvida website|mortigoza}}
-{{Review|We created our own analytics and Deepstream (detection) services}}
-{{Review|From NVIDIA we are using VST, Redis and Ingress}}
-'''RidgeRun Microservices:'''
+This system is made up of the following Microservices:
-#'''PTZ Microservice:'''The PTZ Microservice is designed to receive 360-degree equirectangular RTSP video streams and outputs an RTSP stream with the performed PTZ operations over the equirectangular videos using RidgeRun libpanorama.
-#'''Detection Microservice:''' Detection Microservice detects in the input stream the target objects described in a text prompt.The microservice use the NanoOwl generative AI model that allows open vocabulary detection. Meaning that the user can provide a list of objects that are not bounded to any specific classes. For example, it is possible to indicate: search for dogs.
-#'''AI Agent:''' AI Agent Microservice allows a natural communication between the user and other microservices. This service uses the Hugging Face LLM  Trelis/Llama-2-7b-chat-hf-function-calling-v3 to convert text commands into API calls, processes the LLM result and calls the correspoding API request. For example, it is possible to request: move the camera 30 degrees to the Right.
-#'''Analytics:''' This Microservice reads detection metadata provided by the REDIS NVIDIA microservice and moves the camera to the detected object position using PTZ and starts video recordings using the NVIDIA VST microservice.
+'''[https://developer.nvidia.com/blog/bringing-generative-ai-to-the-edge-with-nvidia-metropolis-microservices-for-jetson/ NVIDIA Metropolis Microservices:''']
+#[https://docs.nvidia.com/moj/vst/VST_Overview.html '''Video Storage Toolkit (VST):'''] VST NVIDIA microservice auto-discovers ONVIF-S compliant IP cameras, and allows the use of custom IP stream as video source. It then allows for video to be stored, played back at various speeds, or paused at any frame
+#[https://docs.nvidia.com/moj/platform-services/redis.html '''REDIS: '''] Redis is a message broker that contains all device metadata. VST depends on Redis streams to publish device status change events. DeepStream depends on Redis streams to publish metadata and receive VST camera status events.
+#[https://docs.nvidia.com/moj/platform-services/ingress.html '''INGRESS: '''] Ingress is the API Gateway to all Metropolis Microservices APIs. It acts as a reverse proxy, where incoming API requests pass through it, which it then forwards to the appropriate microservice based on path prefix.
-{{Review|Add links to the respective MS RR wikis|mortigoza}}
-{{Review|Add a diagram|mortigoza}}
-{{Review|Lets remove BIPS from the list as it is not part of the demo}}
-In this demonstration as all the services are containerized, they can be started by using docker compose. This reduce the complexity of starting and configuring everything to a single compose command.
+'''RidgeRun Microservices:'''
+#[https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun_Services/PTZ '''PTZ Microservice:'''] The PTZ Microservice is designed to receive 360-degree equirectangular RTSP video streams and outputs an RTSP stream with the performed PTZ operations over the equirectangular videos using RidgeRun libpanorama.
+#[https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun_Services/Detection '''Detection Microservice:'''] Detection Microservice detects in the input stream the target objects described in a text prompt.The microservice uses the NanoOwl generative AI model that allows open vocabulary detection. Meaning that the user can provide a list of objects that are not bound to any specific classes. For example, it is possible to indicate: search for dogs.
+#[https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun_Services/AI_Agent '''AI Agent:'''] AI Agent Microservice allows a natural communication between the user and other microservices. This service uses the Hugging Face LLM  Trelis/Llama-2-7b-chat-hf-function-calling-v3 to convert text commands into API calls, process the LLM result, and call the corresponding API request. For example, it is possible to request: move the camera 30 degrees to the Right.
+#[https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun_Services/Analytics '''Analytics:'''] This Microservice reads detection metadata provided by the REDIS NVIDIA microservice and moves the camera to the detected object position using PTZ and starts video recordings using the NVIDIA VST microservice.
-...........
 <noinclude>
-{{Metropolis Microservices/Foot||}}
+{{Metropolis Microservices/Foot
+|previous=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Microservices Demo
+|next=https://developer.ridgerun.com/wiki/index.php/Metropolis_Microservices/RidgeRun Microservices Demo/Getting the code
+}}
 </noinclude>