RidgeRun Developer Wiki

R2Inference/Supported backends/TensorFlow-Lite: Difference between revisions

Page Discussion

R2Inference/Supported backends/TensorFlow-Lite (view source)

Revision as of 20:17, 8 December 2020

10 bytes added ,  8 December 2020
m
no edit summary
No edit summary
mNo edit summary
Line 6: Line 6:
-->
-->


TensorFlow Lite is an open source software library that is part of TensorFlow™. This provides a deep learning framework for on-device inference. Tensorflow lite models can be used on Android and IOS, also can be used on systems like Raspberry Pi and Arm64-based boards. The TensorFlow lite backend supports <code>.tflite</code> models and <code>.tflite</code> quantized models.
TensorFlow Lite is an open-source software library that is part of TensorFlow™. This provides a deep learning framework for on-device inference. Tensorflow lite models can be used on Android and IOS, also can be used on systems like Raspberry Pi and Arm64-based boards. The TensorFlow lite backend supports <code>.tflite</code> models and <code>.tflite</code> quantized models.
==Installation==
==Installation==


R2Inference TensorFlow Lite backend depends on the C/C++ TensorFlow API. The installation process consists on downloading the source code, build and install it.  
R2Inference TensorFlow Lite backend depends on the C/C++ TensorFlow API. The installation process consists on downloading the source code, build and install it.  


TensorFlow python API and utilities can be installed with python pip. These '''are not needed''' by R2Inference, but they are highly recommended if you need to generate models.
TensorFlow Python API and utilities can be installed with Python pip. These '''are not needed''' by R2Inference, but they are highly recommended if you need to generate models.


===X86===
===X86===
Line 125: Line 125:
# Save the graph structure that describes your model
# Save the graph structure that describes your model
# Save checkpoint of your model training session (Session variables)
# Save checkpoint of your model training session (Session variables)
# Combine the graph structure with the checkpoint data (this step is typically refer to as freezing the graph)
# Combine the graph structure with the checkpoint data (this step is typically referred to as freezing the graph)
=== Saving a session with TensorFlow python API===
=== Saving a session with TensorFlow python API===


In Tensorflow, you can use a '''saver''' object to handle saving and restoring of your model graph metadata and the checkpoint (variables) data. In general terms, outside a Tensorflow '''session''' a graph contains only the information regarding the mathematical operation that is performed, while the variables are given a particular value inside a a session. Tipically, after training you model, you can the us a '''saver''' object to save both your graph structure and data checkpoint. The following is an example when working on the Tensorflow default graph:
In Tensorflow, you can use a '''saver''' object to handle saving and restoring of your model graph metadata and the checkpoint (variables) data. In general terms, outside a Tensorflow '''session''' a graph contains only the information regarding the mathematical operation that is performed, while the variables are given a particular value inside a session. Typically, after training your model, you can use a '''saver''' object to save both your graph structure and data checkpoint. The following is an example when working on the Tensorflow default graph:


<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
Line 157: Line 157:
* '''model_graph.chkp.index''': This file has a key-value table linking a tensor name and the location to find the corresponding data in the chkp.data files
* '''model_graph.chkp.index''': This file has a key-value table linking a tensor name and the location to find the corresponding data in the chkp.data files
* '''model_graph.chkp.data-00000-of-00001''': Holds all variables (which includes weights of the graph) from the session at different timestamps
* '''model_graph.chkp.data-00000-of-00001''': Holds all variables (which includes weights of the graph) from the session at different timestamps
* '''checkpoint''' : A file that keeps a record of latest checkpoint files saved
* '''checkpoint''' : A file that keeps a record of the latest checkpoint files saved


The most important files are the chkp.meta and chkp.data files. On a directory with such files you can use the [https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py freeze_graph.py ] method provided by Tensorflow resources on a directory with the files generated by the '''saver''' object in order to generate a protocol buffer file suitable for GstInference.  
The most important files are the chkp.meta and chkp.data files. On a directory, with such files, you can use the [https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py freeze_graph.py ] method provided by Tensorflow resources on a directory with the files generated by the '''saver''' object in order to generate a protocol buffer file suitable for GstInference.  


You can refer to [https://www.ridgerun.com/store/Deep-Learning-Models-and-Binaries-c33344794 R2Inference Model Zoo] for pre-trained models suitable for evaluating GstInference.
You can refer to [https://www.ridgerun.com/store/Deep-Learning-Models-and-Binaries-c33344794 R2Inference Model Zoo] for pre-trained models suitable for evaluating GstInference.
=== Create a model using saved weights from a saved model ===
=== Create a model using saved weights from a saved model ===


This example code take and saved model from a directory and convert it to Tensorflow lite model with .tflite extension.
This example code takes and saved model from a directory and converts it to a Tensorflow lite model with .tflite extension.


<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
Line 207: Line 207:
===Tensorboard===
===Tensorboard===


TensorBoard is a visualization tool for TensorFlow. You can use TensorBoard to visualize your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it. To use TensorBoard you simply need to [https://www.tensorflow.org/install/ install TensorFlow core]. Installing TensorFlow via pip should also automatically install TensorBoard. This tool is specially useful to determine the input and output layer name of undocumented graphs. TensorBoard can load any TensorFlow checkpoint generated with the same version (loading a checkpoint generated with a different Tensorflow version will result on errors).
TensorBoard is a visualization tool for TensorFlow. You can use TensorBoard to visualize your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it. To use TensorBoard you simply need to [https://www.tensorflow.org/install/ install TensorFlow core]. Installing TensorFlow via pip should also automatically install TensorBoard. This tool is specially useful to determine the input and output layer name of undocumented graphs. TensorBoard can load any TensorFlow checkpoint generated with the same version (loading a checkpoint generated with a different Tensorflow version will result in errors).
<syntaxhighlight lang=bash>
<syntaxhighlight lang=bash>
tensorboard --logdir=route/to/checkpoint/dir
tensorboard --logdir=route/to/checkpoint/dir
Line 219: Line 219:


==API==
==API==
You can find the full documentation of the C API [https://www.tensorflow.org/guide/extend/cc here] and the Python API [https://www.tensorflow.org/api_docs/python/ here]. R2Inference uses only the C API and R2Inference takes care of the session, loading the graph and executing. Because of this, we will only take a look at the options that you can change when using the C API through R2Inference.
You can find the full documentation of the C API [https://www.tensorflow.org/guide/extend/cc here] and the Python API [https://www.tensorflow.org/api_docs/python/ here]. R2Inference uses only the C API and R2Inference takes care of the session, loading the graph, and executing. Because of this, we will only take a look at the options that you can change when using the C API through R2Inference.


R2Inference changes the options of the framework via the "IParameters" class. First you need to create an object:
R2Inference changes the options of the framework via the "IParameters" class. First you need to create an object:
Line 255: Line 255:
| Integer
| Integer
| R/W
| R/W
| Allow the usage of 16 bit float point instead of 32 bits
| Allow the usage of 16-bit float point instead of 32 bits


|}
|}
Retrieved from "https://developer.ridgerun.com/wiki/index.php/Special:MobileDiff/33798"