Tensorflow Lite Example

Introduction to Tensorflow Lite

This wiki shows a tutorial of how to build, install and use the Tensorflow Lite API with C++.

Build and install Tensorflow Lite

Download Tensorflow source code:

git clone https://github.com/tensorflow/tensorflow
cd tensorflow/tensorflow/lite/tools/make

For the EdgeTPU example, you need to checkout this TENSORFLOW COMMIT that is compatible with the EdgeTPU library.

Download dependencies:

./download_dependencies.sh

Build:

./build_lib.sh

Copy the static library to the libraries path:

cp gen/linux_x86_64/lib/libtensorflow-lite.a /usr/lib/x86_64-linux-gnu/

Troubleshooting

If the commit of Tensorflow needed to build the EdgeTPU example is d855adfc5a0195788bf5f92c3c7352e638aa1109, you may have the following issue:

/home/lmurillo/work/ridgerun/ai/backends/tensorflow-edgetpu/tensorflow/lite/tools/make/gen/aarch64_armv8-a/lib/libtensorflow-lite.a(densify.o): In function `tflite::ops::builtin::densify::Eval(TfLiteContext*, TfLiteNode*)':
densify.cc:(.text+0x378): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x384): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::SparseToDense(signed char const*)'
densify.cc:(.text+0x5f0): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x5fc): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::SparseToDense(float const*)'

To solve it apply the following change in your local Tensorflow repository:

diff --git a/tensorflow/lite/tools/make/Makefile b/tensorflow/lite/tools/make/Makefile
index 4219aa5c31..43ded75fd6 100644
--- a/tensorflow/lite/tools/make/Makefile
+++ b/tensorflow/lite/tools/make/Makefile
@@ -110,6 +110,7 @@ $(wildcard tensorflow/lite/kernels/*.cc) \
 $(wildcard tensorflow/lite/kernels/internal/*.cc) \
 $(wildcard tensorflow/lite/kernels/internal/optimized/*.cc) \
 $(wildcard tensorflow/lite/kernels/internal/reference/*.cc) \
+$(wildcard tensorflow/lite/tools/optimize/sparsity/*.cc) \
 $(PROFILER_SRCS) \
 tensorflow/lite/tools/make/downloads/farmhash/src/farmhash.cc \
 tensorflow/lite/tools/make/downloads/fft2d/fftsg.c \

Install absl (Tensorflow dependence)

Download absl source code

git clone https://github.com/abseil/abseil-cpp

Build:

cd abseil-cpp
mkdir build && cd build
cmake .. -DABSL_RUN_TESTS=ON -DABSL_USE_GOOGLETEST_HEAD=ON -DCMAKE_CXX_STANDARD=11
cmake --build . --target all
sudo make install

Example

To run the examples first clone this project:

git clone https://gitlab.com/RidgeRun/tflite-example.git
cd tflite-example

TFlite

Modify the Makefile to add the correct Tensorflow path.

Build the code:

make

Finally run the example:

./tflite_example --tflite_model models/mobilenet_v1_1.0_224.tflite --labels models/labels.txt --image testdata/dog.jpg -v 0

Using the mobilenet_v1_1.0_224.tflite the resutl will be:

0.915627: 209 209:Labrador retriever
0.0428241: 208 208:golden retriever
0.00945397: 183 183:Border terrier
0.00348756: 186 186:Norfolk terrier
0.00233: 187 187:Norwich terrier

Using the mobilenet_v1_1.0_224_quant.tflite the resutl will be:

0.901961: 209 209:Labrador retriever
0.054902: 208 208:golden retriever
0.0117647: 183 183:Border terrier
0.00392157: 244 244:bull mastiff
0.00392157: 201 201:Tibetan terrier, chrysanthemum dog

EdgeTPU

Modify the Makefile to add the correct Tensorflow and EdgeTPU paths.

Build the code:

make

Finally run the example:

./edgetpu_example <model path> <image path>

Using the mobilenet_v2_1.0_224_quant_edgetpu.tflite the resutl will be:

$ ./edgetpu_example
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 14.174 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 3.287 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 3.766 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 2.851 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 2.846 ms
=====================================

Using the mobilenet_v2_1.0_224_quant.tflite the result will be:

$ ./edgetpu_example ../models/mobilenet_v2_1.0_224_quant.tflite ../testdata/resized_cat.bmp 
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 143.926 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.345 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.401 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.433 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.485 ms
=====================================

The mobilenet_v2_1.0_224_quant.tflite model is not compatible with the TPU, hence the inferences are performed in the CPU.

For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.

Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.

❯