Tensorflow Lite Example

Introduction to Tensorflow Lite

This wiki shows a tutorial of how to build, install and use the Tensorflow Lite API with C++.

Build and install Tensorflow Lite

Download Tensorflow source code:

git clone https://github.com/tensorflow/tensorflow
cd tensorflow/tensorflow/lite/tools/make

For the EdgeTPU example, you need to checkout this TENSORFLOW COMMIT that is compatible with the EdgeTPU library.

Download dependencies:

./download_dependencies.sh

Build:

./build_lib.sh

Copy the static library to the libraries path:

cp gen/linux_x86_64/lib/libtensorflow-lite.a /usr/lib/x86_64-linux-gnu/

Troubleshooting

If the commit of Tensorflow needed to build the EdgeTPU example is d855adfc5a0195788bf5f92c3c7352e638aa1109, you may have the following issue:

/home/lmurillo/work/ridgerun/ai/backends/tensorflow-edgetpu/tensorflow/lite/tools/make/gen/aarch64_armv8-a/lib/libtensorflow-lite.a(densify.o): In function `tflite::ops::builtin::densify::Eval(TfLiteContext*, TfLiteNode*)':
densify.cc:(.text+0x378): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x384): undefined reference to `tflite::optimize::sparsity::FormatConverter<signed char>::SparseToDense(signed char const*)'
densify.cc:(.text+0x5f0): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::FormatConverter(std::vector<int, std::allocator<int> > const&, TfLiteSparsity const&)'
densify.cc:(.text+0x5fc): undefined reference to `tflite::optimize::sparsity::FormatConverter<float>::SparseToDense(float const*)'

To solve it apply the following change in your local Tensorflow repository:

diff --git a/tensorflow/lite/tools/make/Makefile b/tensorflow/lite/tools/make/Makefile
index 4219aa5c31..43ded75fd6 100644
--- a/tensorflow/lite/tools/make/Makefile
+++ b/tensorflow/lite/tools/make/Makefile
@@ -110,6 +110,7 @@ $(wildcard tensorflow/lite/kernels/*.cc) \
 $(wildcard tensorflow/lite/kernels/internal/*.cc) \
 $(wildcard tensorflow/lite/kernels/internal/optimized/*.cc) \
 $(wildcard tensorflow/lite/kernels/internal/reference/*.cc) \
+$(wildcard tensorflow/lite/tools/optimize/sparsity/*.cc) \
 $(PROFILER_SRCS) \
 tensorflow/lite/tools/make/downloads/farmhash/src/farmhash.cc \
 tensorflow/lite/tools/make/downloads/fft2d/fftsg.c \

Install absl (Tensorflow dependence)

Download absl source code

git clone https://github.com/abseil/abseil-cpp

Build:

cd abseil-cpp
mkdir build && cd build
cmake .. -DABSL_RUN_TESTS=ON -DABSL_USE_GOOGLETEST_HEAD=ON -DCMAKE_CXX_STANDARD=11
cmake --build . --target all
sudo make install

Example

To run the examples first clone this project:

git clone https://gitlab.com/RidgeRun/tflite-example.git
cd tflite-example

TFlite

Modify the Makefile to add the correct Tensorflow path.

Build the code:

make

Finally run the example:

./tflite_example --tflite_model models/mobilenet_v1_1.0_224.tflite --labels models/labels.txt --image testdata/dog.jpg -v 0

Using the mobilenet_v1_1.0_224.tflite the resutl will be:

0.915627: 209 209:Labrador retriever
0.0428241: 208 208:golden retriever
0.00945397: 183 183:Border terrier
0.00348756: 186 186:Norfolk terrier
0.00233: 187 187:Norwich terrier

Using the mobilenet_v1_1.0_224_quant.tflite the resutl will be:

0.901961: 209 209:Labrador retriever
0.054902: 208 208:golden retriever
0.0117647: 183 183:Border terrier
0.00392157: 244 244:bull mastiff
0.00392157: 201 201:Tibetan terrier, chrysanthemum dog

EdgeTPU

Modify the Makefile to add the correct Tensorflow and EdgeTPU paths.

Build the code:

make

Finally run the example:

./edgetpu_example <model path> <image path>

Using the mobilenet_v2_1.0_224_quant_edgetpu.tflite the resutl will be:

$ ./edgetpu_example
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 14.174 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 3.287 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 3.766 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 2.851 ms
=====================================
[Image analysis] max value index: 286 value: 0.773438
invoked
average time: 2.846 ms
=====================================

Using the mobilenet_v2_1.0_224_quant.tflite the result will be:

$ ./edgetpu_example ../models/mobilenet_v2_1.0_224_quant.tflite ../testdata/resized_cat.bmp 
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 143.926 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.345 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.401 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.433 ms
=====================================
[Image analysis] max value index: 286 value: 0.792969
invoked
average time: 142.485 ms
=====================================

The mobilenet_v2_1.0_224_quant.tflite model is not compatible with the TPU, hence the inferences are performed in the CPU.

RidgeRun Resources