GstInference Smart Lock application

GstInference
Introduction
Getting started
Getting the code Building the plugin Quick Starting Guide
Supported architectures
InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3 YoloV2 ResNetV1 MobileNetV2 MobileNetV2 + SSD AlexNet
Supported backends
TensorFlow Tensorflow-Lite Coral from Google Caffe TensorRT ONNXRT ONNXRT ACL ONNXRT OpenVINO
Metadata and Signals
GstClassificationMeta GstEmbeddingMeta GstDetectionMeta GstInferenceMeta Signals
Overlay Elements
Inference Overlay
Utils Elements
InferenceFilter InferenceCrop InferenceDebug InferenceBin
Legacy pipelines
PC NANO TX2 Xavier IMX8
Example pipelines
Example applications
Classification Detection Hand Pose Recognition
Benchmarks
Model Zoo
Project Status
Releases Roadmap
Contact Us

General information

This application uses 2 programs to create a Smart Lock using a folder with valid faces and a camera to feed a FaceNetV1 network. The valid faces will be used to create a file with valid metadata, the camera will be sending streaming to a FaceNetV1 network to create metadata related to the current frame. The metadata will be compared to choose if the face is valid or not.

Application

The application is separated into 2 different programs:

• facedetection_train.py: Uses JPEG images from a selected folder to create the metadata file with valid faces.
• facedetection_run.py: Uses the metadata and labels with valid faces to choose if a face (from the camera streaming) is valid.

Embeddings and labels example

The application uses a images/ directory with images (provided in the same application folder) to create the metadata and labels file.

• embeddings.txt example:

-0.0145593 -0.0584495 0.0371784 -0.0912583 0.0490291 -0.0138151 -0.0323807 -0.0655067 0.0632302 -0.108694 0.13243 -0.0735193 0.00284548 -0.136761 -0.0199185 0.0698619 0.126645 0.1193 0.0110647 -0.111995 0.0158512 0.00433643 -0.0758559 0.0262919 0.0479901 0.0946619 0.0865307 0.117623 -0.0716868 -0.00987446 0.02357 0.153645 -0.0515243 0.0114069 0.0752071 -0.0052865 0.0569346 0.185266 0.13548 0.0183175 0.0196599 -0.0376986 0.125624 -0.163115 -0.081066 -0.0181404 0.0306145 -0.161885 0.0324361 -0.025209 0.0890772 0.0985957 0.169964 -0.0656467 0.132334 0.197022 -0.130948 -0.128935 0.0770448 -0.0759822 0.0719929 0.041964 -0.0149276 0.100265 0.0512894 0.0797284 0.0842571 0.00714669 -0.0802012 0.126848 0.102854 -0.0279055 0.0943478 -0.141276 -0.0672122 0.0512549 -0.0887869 0.00805435 -0.138771 -0.154171 0.0184849 -0.00798414 -0.0382218 0.262642 0.0620274 0.0160168 0.131385 -0.173691 -0.0936234 -0.0327744 -0.00891561 -0.0502745 -0.0983856 -0.0451589 0.0597074 -0.0466168 0.0568609 -0.0749385 0.0241126 -0.0180538 0.0295493 0.0973359 0.0317965 -0.121993 -0.000457776 0.162942 -0.127862 0.033625 -0.0521476 -0.022432 0.068894 0.0548606 0.0497928 0.0185768 0.0157577 0.0550619 -0.143579 0.0650198 -0.135888 -0.0184038 0.010344 -0.0246181 0.183069 0.0519738 0.00459688 -0.0259964 0.111819 -0.0981886;-0.0242322 0.00290796 -0.162734 0.0286987 0.0642769 -0.0359746 -0.0148205 -0.0628836 -0.116573 0.0260048 0.0582735 0.0380352 0.028675 -0.105973 0.0646818 0.0244191 0.0484146 -0.0989158 -0.0378393 0.135033 0.141195 -0.105727 -0.00524461 0.0412349 -0.0236515 -0.00295492 0.0143279 0.0563888 0.0473259 0.0471952 0.0906711 0.0400181 -0.0451043 0.04677 -0.141052 -0.156074 0.102902 -0.139874 -0.0758197 -0.0876225 0.0337217 0.0562246 0.047203 0.0574563 -0.0346048 0.0272441 0.1464 -0.11729 -0.0192566 0.0500386 -0.20818 0.00922727 -0.0418217 -0.0439746 -0.0460908 -0.0146872 0.128959 0.145776 -0.010307 -0.0502751 -0.0336426 0.0830565 0.151228 0.144064 -0.21205 0.0717333 0.0502196 -0.0608728 0.108237 -0.0956707 0.120978 -0.178943 -0.0360214 0.0257333 0.0886595 0.0511544 0.0311828 0.0356794 -0.00302341 -0.0439167 -0.0331555 -0.0353495 -0.0187001 -0.100847 0.0497707 0.179336 -0.164382 0.10775 0.00772931 -0.0325079 0.0483728 -0.195318 0.0763725 0.0769976 0.162546 0.0145742 -0.029331 -0.0993435 -0.148174 -0.0178964 -0.0752645 0.081258 0.128173 -0.0234716 0.0159863 -0.119493 0.0897941 0.0113501 -0.0650295 0.112933 -0.14094 0.12612 0.0619066 0.0242797 -0.106637 -0.0690303 -0.142767 -0.0396135 -0.0697015 -0.0323864 0.0244745 0.0639094 0.124145 -0.130794 0.0173324 -0.11673 0.116843 0.0512073

• labels.txt example:

Barack_Obama;Chuck_Norris

Output Example using pipelines

See examples below using the metadata in the section above.

Using a face that is not valid will display a fail message.

Face not valid in the metadata

In order to detect new faces, a JPEG image can be saved into images directory. After adding images, run facedetection_train.py to create an updated embeddings and labels file. Finally, run facedetection_run.py to execute the Gstreamer pipeline.

Face valid in the metadata after adding a new face to images directory

Usage notes

• TensorFlow for Python is required to create the embeddings metadata.
• FaceNetV1 TensorFlow model can be downloaded from RidgeRun's store.
• Some computers require to reset the locale in a terminal to correctly parse the embeddings metadata, run this command if you can not replicate the output example:

export LC_ALL="en_US.UTF-8"

Reporting a Bug

Please feel free to report bugs using GitHub's issue tracker.