NVIDIA Jetson Xavier - Jetson Reinforcement Deep Learning Tutorial

NVIDIA®Jetson Xavier™
Introduction
SoM Overview Carrier Boards NVIDIA Xavier Developer Kit Connect Tech Rogue Connect Tech Mimic Software Support NVIDIA JetPack
JetPack 5.X
Getting Started Installing JetPack Flashing the board with GUI installer JetPack Components Compiling Source Code Flashing the Board from Cmdline Performance Tuning Evaluating Performance Set Values Manually Tuning Power Maximizing Performance
Development in the Board
Getting into the Board Using Serial Console Using SSH Using Ubuntu Login Installing Packages
Video Capture and Display
Description Supported Cameras Imx219 Driver Software Support Using GStreamer Using Libargus
Processors
HD Audio and Video Subystem Video Encoder Video Decoder JPEG Encoder and Decoder Video Image Compositor Audio Processing Engine Deep Learning Accelerator Volta GPU Description Using CUDA Using OpenGL
Interfaces
I2C SD/eMMC Controller USB Controller
GStreamer Pipelines
Capture and Display H264 H265 VP9 MPEG4 Simultaneous Streams Compositor
Deep Learning
TensorRT Using TensorRT integrated with Tensorflow Parsing a Tensorflow model for TensorRT Parsing Caffe model for TensorRT Building TensorRT API examples Deep Learning Accelerator Software Support Deep Learning Tutorials Jetson Inference Jetson Reinforcement
RidgeRun Products
GstPTZR GstCUDA GstColorTransfer GstWebRTC GstRtspSink GstInterpipe GstInference GStreamer Daemon GstShark
Reference Documentation
Contact Us

Jetson-reinforcement is a training guide for deep reinforcement learning on the TX1 and TX2 using PyTorch. The tutorial is not currently supported on the Jetson Xavier. These instructions will help you test the first example described on the repository without using it directly. To view the full training you can visit the Github repository.

Reinforcement learning stack

Installing requirements

Prerequisites

Install prerequisites

sudo apt-get install python-pip cmake

Upgrade pip

sudo -H pip install -U pip

Hack pip for Ubuntu 18.0 by changing file '/usr/bin/pip' with the following diff:

diff --git a/pip b/pip
index 56bbb2b..62f26b9 100755
--- a/pip
+++ b/pip
@@ -6,6 +6,6 @@ import sys
 # Run the main entry point, similarly to how setuptools does it, but because
 # we didn't install the actual entry point from setup.py, don't use the
 # pkg_resources API.
-from pip import main
+from pip import __main__
 if __name__ == '__main__':
-    sys.exit(main())
+    sys.exit(__main__._main())

Gazebo

The Gazebo is a robotics simulator and it's used on a robotic arm example that is not yet available.

Robotic arm simulation running on Gazebo

To install Gazebo run

sudo apt-get install curl
curl -sSL http://get.gazebosim.org | sh

Gazebo will throw an error on the Xavier because it looks for a broken link. You can fix it with:

sudo rm /usr/lib/aarch64-linux-gnu/libdrm.so.2 #remove the broken link
sudo -H ln -s /usr/lib/aarch64-linux-gnu/libdrm.so.2.4.0 /usr/lib/aarch64-linux-gnu/libdrm.so.2

Test gazebo

gazebo

PyTorch

With this instruction, you will install PyTorch v1.0.0. This is the only version of PyTorch that supports CUDA 10. However, the demos on the jetson-reinforcement repository are programmed with PyTorch 0.3 and the library syntax changed considerably on version 0.4. At the moment only the first demo has been ported to version 1.0.0, the other ones will be available in the future [1].

Download wheel file from here and install it with pip:

sudo -H pip install torch-1.0.0a0+8601b33-cp27-cp27mu-linux_aarch64.whl
sudo -H pip install numpy

There is also a python 3.6 wheel here

Test pytorch installation:

import torch
print(torch.__version__)
print('CUDA available: ' + str(torch.cuda.is_available()))
a = torch.cuda.FloatTensor(2).zero_()
print('Tensor a = ' + str(a))
b = torch.randn(2).cuda()
print('Tensor b = ' + str(b))
c = a + b
print('Tensor c = ' + str(c))

Your output should look something like this:

>>> import torch
>>> print(torch.__version__)
1.0.0a0+b7a7ab3
>>> print('CUDA available: ' + str(torch.cuda.is_available()))
CUDA available: True
>>> import torch
>>> print(torch.__version__)
1.0.0a0+b7a7ab3
>>> print('CUDA available: ' + str(torch.cuda.is_available()))
CUDA available: True
>>> a = torch.cuda.FloatTensor(2).zero_()
>>> print('Tensor a = ' + str(a))
Tensor a = tensor([0., 0.], device='cuda:0')
>>> b = torch.randn(2).cuda()
>>> print('Tensor b = ' + str(b))
Tensor b = tensor([-0.6599, -1.0124], device='cuda:0')
>>> c = a + b
>>> print('Tensor c = ' + str(c))
Tensor c = tensor([-0.6599, -1.0124], device='cuda:0')

Deep Q Learning + OpenAI Gym

Cartpole

This tutorial shows how to use PyTorch to train a Deep Q Learning (DQN) agent on the CartPole-v0 task from the OpenAI Gym.

Cartpole example

In this problem, a pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1(1.0) or -1(0.0) to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of 1.0 is provided for every timestep that the pole remains upright and the reward is reduced to 0.0 once the pendulum surpasses the recovery angle (12 degrees) or the cart moves more than 2.4 units from the center.

For this problem, the environment provides 4 observations: Cart Position, Cart Velocity, Pole Angle, and Pole Velocity At Tip. However, the Deep Q learning approach to this problem only uses captures (images) of the environment to train a neural network and solve the problem.

You can get the jupyter notebook here. You can get the python code here.

To run the example:

# Jupyter version
jupyter notebook reinforcement_q_learning.ipynb
# Python version
python reinforcement_q_learning.py

C++ API

This example is not yet available

3D Simulation

This example is not yet available

Continuous Control

This example is not yet available