NVIDIA Jetson Xavier - Using TensorRT integrated with Tensorflow

NVIDIA®Jetson Xavier™
Introduction
SoM Overview Carrier Boards NVIDIA Xavier Developer Kit Connect Tech Rogue Connect Tech Mimic Software Support NVIDIA JetPack
JetPack 5.X
Getting Started Installing JetPack Flashing the board with GUI installer JetPack Components Compiling Source Code Flashing the Board from Cmdline Performance Tuning Evaluating Performance Set Values Manually Tuning Power Maximizing Performance
Development in the Board
Getting into the Board Using Serial Console Using SSH Using Ubuntu Login Installing Packages
Video Capture and Display
Description Supported Cameras Imx219 Driver Software Support Using GStreamer Using Libargus
Processors
HD Audio and Video Subystem Video Encoder Video Decoder JPEG Encoder and Decoder Video Image Compositor Audio Processing Engine Deep Learning Accelerator Volta GPU Description Using CUDA Using OpenGL
Interfaces
I2C SD/eMMC Controller USB Controller
GStreamer Pipelines
Capture and Display H264 H265 VP9 MPEG4 Simultaneous Streams Compositor
Deep Learning
TensorRT Using TensorRT integrated with Tensorflow Parsing a Tensorflow model for TensorRT Parsing Caffe model for TensorRT Building TensorRT API examples Deep Learning Accelerator Software Support Deep Learning Tutorials Jetson Inference Jetson Reinforcement
RidgeRun Products
GstPTZR GstCUDA GstColorTransfer GstWebRTC GstRtspSink GstInterpipe GstInference GStreamer Daemon GstShark
Reference Documentation
Contact Us

Using TensorRT integrated with Tensorflow

TensorFlow is an open-source software library for numerical computation using data flow graphs. The graph nodes represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. Nowadays Tensorflow is primarily used as a framework to easily develop and deploy deep learning solutions.

Starting with version 1.7 Tensorflow supports native TensorRT, which can improve latency and throughput for inference for some models. This section provides a guide for getting Tensorflow up and running on Xavier with TensorRT support.

At the end of this guide you will be able to:

• Train deep learning models on Xavier using Tensorflow with native TensorRT support

To follow this guide you will need:

• Jetson Xavier with JetPack 4.1
• Tensorflow wheel for JetPack 4.1
• Official ResNet model

Step 1: Install Prerequisites

• Install JetPack with TensorRT(5.0), CUDA Toolkit(10.0) and cuDNN.

• Install dependencies:

sudo apt-get install zip unzip autoconf automake libtool curl zlib1g-dev maven -y
sudo apt-get install python-numpy swig python-dev python-pip python-wheel -y

Step 2: Install Tensorflow

Download the TensorFlow wheel file for JetPack 4.0 and Install it with the following command:

sudo -H pip install tensorflow-1.10.1-cp27-cp27mu-linux_aarch64.whl

To validate your Tensorflow installatio, start a terminal and invoke python:

python

Enter the following short program using the python interactive shell:

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

If the system outputs the following, then you are ready to begin writing TensorFlow programs:

Hello, TensorFlow!

Step 3: Train a model

Clone the official tensorflow models repository and add it to the python path and install prerequisites:

git clone https://github.com/tensorflow/models.git
export PYTHONPATH="$PYTHONPATH:/path/to/models"
pip install --user -r models/official/requirements.txt

Then download and extract the CIFAR-10 data from Alex's website, specifying the location with the --data_dir flag:

python models/official/resnet/cifar10_download_and_extract.py

Then to train the model, run the following:

python models/official/resnet/cifar10_main.py

The resulting checkpoint file will be saved under /tmp/cifar10_model/. For more information on how to train a specific model please refer to the Tensorflow tutorials.