Coral from Google/GstInference/Example Pipelines: Difference between revisions

From RidgeRun Developer Wiki
mNo edit summary
mNo edit summary
 
(16 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<noinclude>
<noinclude>
{{Coral from Google/Head|next=GstInference/Demos|previous=GstInference/Why_use_GstInference?|keywords=}}
{{Coral from Google/Head|next=GstInference/Demos|previous=GstInference/Why_use_GstInference?|metakeywords=}}
</noinclude>
</noinclude>


=Introduction=
==Introduction==


The pipelines in this wiki are designed to test the GstInference capabilities in a simple way, so you just need to copy and paste the code inside the colored boxes into your terminal. The blue pipelines are meant to be executed inside the folder that contains the inference model data. The purple pipelines are for displaying the received stream, so they can be executed at any location.
The pipelines in this wiki are designed to test the GstInference capabilities in a simple way, so you just need to copy and paste the code inside the colored boxes into your terminal. The blue pipelines are meant to be executed inside the folder that contains the inference model data. The purple pipelines are for displaying the received stream, so they can be executed at any location.


The model for these pipelines can be downloaded from Google Coral store:
The model and labels for these pipelines can be downloaded from:


* [https://github.com/google-coral/test_data/raw/master/mobilenet_v2_1.0_224_quant_edgetpu.tflite MobilenetV2 (model)]
* '''MobilenetV2''': [https://coral.ai/models/ model and labels].
* '''MobilenetV2 + SSD''': [https://coral.ai/models/ model] and [[Coral_MobilenetV2SSD_COCO_labels | labels]]. In this case, you need to save the labels content into a file named ''coco_labels.txt''.


And for the labels file you may use the one that comes for the tensorflow backend in RidgeRun store:
<pre style="background-color:#f5e05b; width:750px">
* [https://shop.ridgerun.com/products/mobilenetv2-for-tensorflow MobilenetV2 (labels)]
    Important: Make sure you use RidgeRun labels to get the correct inference results.
</pre>


Once you have downloaded them, unzip them and test your preferred pipeline from the list below.
Once you have downloaded them, test your preferred pipeline from the list below.


=Dev Board=
{{Ambox
|type=notice
|small=left
|issue='''Note''': These pipelines have been tested using the Coral USB Accelerator and the Coral Dev Board.
|style=width:unset;
}}


== Classification: MobilenetV2 ==
== Classification: MobilenetV2 ==
Line 22: Line 29:
=== Camera Source ===
=== Camera Source ===


For these pipelines you can modify the CAMERA variable according to your device.
For these pipelines, you can modify the CAMERA variable according to your device.


'''Display Output'''
'''Display Output'''
Line 33: Line 40:
LABELS='imagenet_labels.txt'
LABELS='imagenet_labels.txt'
gst-launch-1.0 \
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! videoconvert ! tee name=t \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! \
videoconvert ! waylandsink fullscreen=false sync=false
waylandsink fullscreen=false sync=false
</pre>
</pre>


Line 51: Line 57:
OUTPUT_FILE='recording.mpeg'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! videoconvert ! tee name=t \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
</pre>
</pre>


Line 70: Line 75:
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
LABELS='imagenet_labels.txt'
HOST='192.168.0.17'
HOST='192.168.0.13'
PORT='5000'
PORT='5000'
gst-launch-1.0 \
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! videoconvert ! tee name=t \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! avenc_mpeg2video ! mpegtsmux ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
udpsink host=$HOST port=$PORT sync=false
</pre>
</pre>


* Client side
* Client-side


<pre style="background:#e4d2fa">
<pre style="background:#e4d2fa">
Line 90: Line 94:
=== File Source ===
=== File Source ===


For these pipelines you can modify the VIDEO_FILE variable in order to provide an mp4 video file that contains any class of the ones listed inside the ''imagenet_labels.txt'' from the downloaded model.
For these pipelines, you can modify the VIDEO_FILE variable in order to provide an mp4 video file that contains any class of the ones listed inside the ''imagenet_labels.txt'' from the downloaded model.


'''Display Output'''
'''Display Output'''
Line 101: Line 105:
LABELS='imagenet_labels.txt'
LABELS='imagenet_labels.txt'
gst-launch-1.0 \
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! videoconvert ! tee name=t \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! \
videoconvert ! waylandsink fullscreen=false sync=false
waylandsink fullscreen=false sync=false
</pre>
</pre>


Line 121: Line 124:
OUTPUT_FILE='recording.mpeg'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! videoconvert ! tee name=t \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
</pre>
</pre>


Line 140: Line 142:
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
LABELS='imagenet_labels.txt'
HOST='192.168.0.17'
HOST='192.168.0.13'
PORT='5000'
PORT='5000'
gst-launch-1.0 \
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! videoconvert ! tee name=t \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! avenc_mpeg2video ! mpegtsmux ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
udpsink host=$HOST port=$PORT sync=false
</pre>
</pre>


* Client side
* Client-side


<pre style="background:#e4d2fa">
<pre style="background:#e4d2fa">
Line 160: Line 161:
=== RTSP Source ===
=== RTSP Source ===


For these pipelines you may modify the RTSP_URI variable according to your needs.
For these pipelines, you may modify the RTSP_URI variable according to your needs.


'''Display Output'''
'''Display Output'''
Line 171: Line 172:
LABELS='imagenet_labels.txt'
LABELS='imagenet_labels.txt'
gst-launch-1.0 \
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! videoconvert ! tee name=t \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! \
videoconvert ! waylandsink fullscreen=false sync=false
waylandsink fullscreen=false sync=false
</pre>
</pre>


Line 191: Line 191:
OUTPUT_FILE='recording.mpeg'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! videoconvert ! tee name=t \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
</pre>
</pre>


Line 210: Line 209:
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
LABELS='imagenet_labels.txt'
HOST='192.168.0.17'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
</pre>
 
* Client-side
 
<pre style="background:#e4d2fa">
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e
</pre>
 
== Detection: MobilenetV2 + SSD ==
 
=== Camera Source ===
 
For these pipelines, you can modify the CAMERA variable according to your device.
 
'''Display Output'''
 
<pre style="background:#D6E4F1">
CAMERA='/dev/video1'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false
</pre>
 
'''Recording Output'''
 
<pre style="background:#D6E4F1">
CAMERA='/dev/video1'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
</pre>
 
''' Streaming Output'''
 
Remember to modify the HOST and PORT variables according to your own needs.
 
* Processing side
<pre style="background:#D6E4F1">
CAMERA='/dev/video1'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
</pre>
 
* Client-side
 
<pre style="background:#e4d2fa">
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e
</pre>
 
=== File Source ===
 
For these pipelines, you can modify the VIDEO_FILE variable in order to provide an mp4 video file that contains any class of the ones listed inside the ''imagenet_labels.txt'' from the downloaded model.
 
'''Display Output'''
 
<pre style="background:#D6E4F1">
VIDEO_FILE='animals.mp4'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false
</pre>
 
'''Recording Output'''
 
You can modify the OUTPUT_FILE variable to the name you want for your recording.
 
<pre style="background:#D6E4F1">
VIDEO_FILE='animals.mp4'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
</pre>
 
''' Streaming Output'''
 
Remember to modify the HOST and PORT variables according to your own needs.
 
* Processing side
<pre style="background:#D6E4F1">
VIDEO_FILE='animals.mp4'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
HOST='192.168.0.13'
PORT='5000'
PORT='5000'
gst-launch-1.0 \
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! videoconvert ! tee name=t \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
t. ! videoscale ! queue ! net.sink_model \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
t. ! queue ! net.sink_bypass \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
mobilenetv2 name=net model-location=$MODEL_LOCATION backend=edgetpu backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
net.src_bypass ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! avenc_mpeg2video ! mpegtsmux ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
udpsink host=$HOST port=$PORT sync=false
</pre>
</pre>


* Client side
* Client-side


<pre style="background:#e4d2fa">
<pre style="background:#e4d2fa">
Line 228: Line 360:
</pre>
</pre>


== Detection: MobilenetSSD v2 ==
=== RTSP Source ===
 
For these pipelines, you may modify the RTSP_URI variable according to your needs.


= USB Accelerator =
'''Display Output'''


<pre style="background:#D6E4F1">
RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false
</pre>
'''Recording Output'''
You can modify the OUTPUT_FILE variable to the name you want for your recording.
<pre style="background:#D6E4F1">
RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e
</pre>
''' Streaming Output'''
Remember to modify the HOST and PORT variables according to your own needs.
* Processing side
<pre style="background:#D6E4F1">
RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
</pre>
* Client-side
<pre style="background:#e4d2fa">
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e
</pre>




Latest revision as of 11:46, 6 March 2023




Previous: GstInference/Why_use_GstInference? Index Next: GstInference/Demos


Coral from Google
Google Coral
Introduction
GStreamer
GstInference
Camera Drivers
Reference Documentation
Contact Us




Introduction

The pipelines in this wiki are designed to test the GstInference capabilities in a simple way, so you just need to copy and paste the code inside the colored boxes into your terminal. The blue pipelines are meant to be executed inside the folder that contains the inference model data. The purple pipelines are for displaying the received stream, so they can be executed at any location.

The model and labels for these pipelines can be downloaded from:

  • MobilenetV2: model and labels.
  • MobilenetV2 + SSD: model and labels. In this case, you need to save the labels content into a file named coco_labels.txt.
    Important: Make sure you use RidgeRun labels to get the correct inference results.

Once you have downloaded them, test your preferred pipeline from the list below.

Classification: MobilenetV2

Camera Source

For these pipelines, you can modify the CAMERA variable according to your device.

Display Output 

CAMERA='/dev/video1'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false

Recording Output 

CAMERA='/dev/video1'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e

Streaming Output

Remember to modify the HOST and PORT variables according to your own needs.

  • Processing side
CAMERA='/dev/video1'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
  • Client-side
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e

File Source

For these pipelines, you can modify the VIDEO_FILE variable in order to provide an mp4 video file that contains any class of the ones listed inside the imagenet_labels.txt from the downloaded model.

Display Output 

VIDEO_FILE='animals.mp4'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false

Recording Output

You can modify the OUTPUT_FILE variable to the name you want for your recording. 

VIDEO_FILE='animals.mp4'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e

Streaming Output

Remember to modify the HOST and PORT variables according to your own needs.

  • Processing side
VIDEO_FILE='animals.mp4'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
  • Client-side
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e

RTSP Source

For these pipelines, you may modify the RTSP_URI variable according to your needs.

Display Output 

RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false

Recording Output

You can modify the OUTPUT_FILE variable to the name you want for your recording. 

RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e

Streaming Output

Remember to modify the HOST and PORT variables according to your own needs.

  • Processing side
RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='mobilenet_v2_1.0_224_quant_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='imagenet_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2 backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
  • Client-side
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e

Detection: MobilenetV2 + SSD

Camera Source

For these pipelines, you can modify the CAMERA variable according to your device.

Display Output 

CAMERA='/dev/video1'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false

Recording Output 

CAMERA='/dev/video1'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e

Streaming Output

Remember to modify the HOST and PORT variables according to your own needs.

  • Processing side
CAMERA='/dev/video1'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
  • Client-side
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e

File Source

For these pipelines, you can modify the VIDEO_FILE variable in order to provide an mp4 video file that contains any class of the ones listed inside the imagenet_labels.txt from the downloaded model.

Display Output 

VIDEO_FILE='animals.mp4'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false

Recording Output

You can modify the OUTPUT_FILE variable to the name you want for your recording. 

VIDEO_FILE='animals.mp4'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e

Streaming Output

Remember to modify the HOST and PORT variables according to your own needs.

  • Processing side
VIDEO_FILE='animals.mp4'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
filesrc location=$VIDEO_FILE ! qtdemux ! queue ! h264parse ! avdec_h264 ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
  • Client-side
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e

RTSP Source

For these pipelines, you may modify the RTSP_URI variable according to your needs.

Display Output 

RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! waylandsink fullscreen=false sync=false

Recording Output

You can modify the OUTPUT_FILE variable to the name you want for your recording. 

RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
OUTPUT_FILE='recording.mpeg'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! filesink location=$OUTPUT_FILE -e

Streaming Output

Remember to modify the HOST and PORT variables according to your own needs.

  • Processing side
RTSP_URI='rtspt://170.93.143.139/rtplive/1701519c02510075004d823633235daa'
MODEL_LOCATION='ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite'
INPUT_LAYER='input'
OUTPUT_LAYER='MobilenetV2/Predictions/Reshape_1'
LABELS='coco_labels.txt'
HOST='192.168.0.13'
PORT='5000'
gst-launch-1.0 \
rtspsrc location=$RTSP_URI ! rtph264depay ! decodebin ! queue ! \
inferencebin arch=mobilenetv2ssd backend=coral model-location=$MODEL_LOCATION \
input-layer=$INPUT_LAYER output-layer=$OUTPUT_LAYER \
labels="\"$(awk '{$1=""; printf "\%s\;",$0}' $LABELS)\"" overlay=true ! \
videoconvert ! avenc_mpeg2video ! mpegtsmux ! udpsink host=$HOST port=$PORT sync=false
  • Client-side
PORT='5000'
gst-launch-1.0 udpsrc port=$PORT ! queue  ! tsdemux ! mpeg2dec ! queue ! videoconvert ! autovideosink sync=false -e


Previous: GstInference/Why_use_GstInference? Index Next: GstInference/Demos



Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Coral_from_Google/GstInference/Example_Pipelines&oldid=47252"