Compiling Jetson TX2 source code L4T 28.2 and Jetpack 3.2.1

Introduction to compiling Jetson TX1 TX2 source code

In this wiki page you are going to find the instructions to download the source code to rebuild the Jetson TX1/TX2 images using jetpack, several parts of this wiki were based in the document called Start_L4T_Docs.html given by NVIDIA in L4T. This page should be considered a work in progress. These steps were run on Jetson Tegra X1 and Tegra X2 and Jetpack 3.2.1 was used. Additionally, in section Articles Related you can find instructions on how to build using older versions of Jetpack.

You can get the documentation from nvl4t_docs.

Platform

Baseboard: P2371
Jetson module: P2180
Soc: tegra21x and tegra18x

It is assumed that the reader already read the brief Jetson TX1 introduction

Downloading the code

In order to be able to download the source code you should have Jetpack installed, if you already have it, skip the next section and go to Getting bootloader and kernel.

Toolchain

1. You can download the official NVIDIA toolchain for JetPack 3.2.1:

GCC 4.8.5 Tool Chain for 64-bit BSP

Other toolchain versions can be found in this link.

2. Install the toolchain

sudo mkdir /opt/l4t-gcc-toolchain-64-bit-28-2.1
sudo chown $USER:$USER /opt/l4t-gcc-toolchain-64-bit-28-2.1
cd /opt/l4t-gcc-toolchain-64-bit-28-2.1

If you already downloaded the toolchain in step 1:

mv ~/Downloads/gcc-4.8.5-aarch64.tgz ./

If not:

wget https://developer.nvidia.com/embedded/dlc/l4t-gcc-toolchain-64-bit-28-2-ga
mv l4t-gcc-toolchain-64-bit-28-2-ga gcc-4.8.5-aarch64.tgz

Finally decompress the toolchain

tar -xvf gcc-4.8.5-aarch64.tgz

Getting JetPack

Check which is the most recent JetPack release.

1. Download Jetpack 3.2.1 from: https://developer.nvidia.com/embedded/dlc/jetpack-l4t-3_2_1

2. Move binary into installation directory, we recommend /home/$USER/JetPack-L4T-3.2.1

export INSTALLDIR=/home/$USER/JetPack-L4T-3.2.1
mkdir -p $INSTALLDIR
mv JetPack-L4T-3.2.1-linux-x64_b23.run $INSTALLDIR

3. Set the Jetpack binary as executable and set correct permissions.

cd $INSTALLDIR
chmod +x JetPack-L4T-3.2.1-linux-x64_b23.run

4. Install Jetpack.

./JetPack-L4T-3.2.1-linux-x64_b23.run

5. Follow the instructions. You'll be prompted for a devkit to build for (TX1, TX2, TX2i) just select the one you want.

Getting bootloader and kernel

File called Start_L4T_Docs.html inside of Jetpack contains a good description of the packages that it contains. Let's call $DEVDIR the path where your L4T development directory is.

For TX1

export DEVDIR=$INSTALLDIR/64_TX1/Linux_for_Tegra

For TX2

export DEVDIR=$INSTALLDIR/64_TX2/Linux_for_Tegra

Then Open the documentation:

firefox $DEVDIR/../../Start_L4T_Docs.html &

Download the kernel source code

1.1 RECOMMENDED

Download kernel sources from next link:

https://developer.nvidia.com/embedded/dlc/sources-r2821#Getting_kernel

Untar the sources and copy kernel_src.tbz2 to $HOME/JetPack-L4T-3.2.1/64_TX2/Linux_for_Tegra/sources

tar -xf public_sources.tbz2
cd public_release/
cp kernel_src.tbz2 $DEVDIR/sources
cd $DEVDIR/sources
tar -xf kernel_src.tbz2

1.2 Option 2

cd $DEVDIR/
./source_sync.sh

This will download the bootloader and kernel.

When syncing, you'll be asked for a tag, lets use tegra-l4t-r28.2.1 for both Kernel and uboot.

Kernel

In order to compile the kernel please follow these steps

Build Kernel

1) Make sure you have the kernel source code and the toolchain already installed by following the instructions in these sections:

2) Configure environment:

mkdir -p $DEVDIR/images/modules 
mkdir -p $DEVDIR/images/packages
mkdir -p $DEVDIR/images/dtb
export CROSS_COMPILE=/opt/l4t-gcc-toolchain-64-bit-28-2.1/install/bin/aarch64-unknown-linux-gnu-
export KERNEL_MODULES_OUT=$DEVDIR/images/modules
export TEGRA_KERNEL_OUT=$DEVDIR/images
export ARCH=arm64

3) Clean your kernel and configuration

cd $DEVDIR/sources/kernel/kernel-4.4
make mrproper

4) Configure your kernel:

This commands will set the default configuration and open a menu so you can change any of the setting if needed. If not, you can just hit exit after the menu opens.

For TX1

make O=$TEGRA_KERNEL_OUT tegra21_defconfig
make O=$TEGRA_KERNEL_OUT menuconfig

For TX2

make O=$TEGRA_KERNEL_OUT tegra18_defconfig
make O=$TEGRA_KERNEL_OUT menuconfig

NOTE: You'll need ncurses for this command to work. If you have any issue just install the following packages.

sudo apt-get install libncurses5 libncurses5-dev

5) Compile kernel, device tree and modules

make O=$TEGRA_KERNEL_OUT zImage
make O=$TEGRA_KERNEL_OUT dtbs
make O=$TEGRA_KERNEL_OUT modules
make O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=$KERNEL_MODULES_OUT

After this the kernel image will be under:

$DEVDIR/images/arch/arm64/boot/Image

And the devicetree under:

$DEVDIR/images/arch/arm64/boot/dts/*.dtb

6) Create kernel_supplements.tbz2 with kernel modules, this package is needed by Jetpack

cd $DEVDIR/images/modules/lib/modules/
ls .

define a variable with the name of the modules directory

export KERNEL_MODULES_NAME=4.4.38+

fix the symbolic links on the kernel modules directory

cd $DEVDIR/images/modules/lib/modules/$KERNEL_MODULES_NAME
rm build source

create tarball

cd $DEVDIR/images/modules/
tar -cjf kernel_supplements.tbz2 *
mv kernel_supplements.tbz2 $DEVDIR/images/packages

7) Besides creating the modules and the kernel image you need to create again your kernel_headers.tbz2 file (needed by Jetpack). By default when you patch the kernel code and you don't check your changes in the kernel will add a -dirty suffix to the release version (check ls $DEVDIR/images/modules/lib/modules/ for example). For this specific reason you need to generate your headers tarball again changing it the release version.

cd $DEVDIR/kernel
tar -xf kernel_headers.tbz2

Find the name of the headers file folder (in my case linux-headers-4.4.38-tegra):

ls | grep linux-headers
linux-headers-4.4.38-tegra

export KERNEL_HEADERS_NAME=linux-headers-4.4.38-tegra

And rename the folder:

mv $KERNEL_HEADERS_NAME linux-headers-$KERNEL_MODULES_NAME
tar -cjf kernel_headers_custom.tbz2 linux-headers-$KERNEL_MODULES_NAME
mv kernel_headers_custom.tbz2 $DEVDIR/images/packages
rm -rf linux-headers-$KERNEL_MODULES_NAME

8) Create a backup of the images and packages included in Jetpack

mkdir -p $DEVDIR/images/packages-backup
cp -rf $DEVDIR/kernel/* $DEVDIR/images/packages-backup

9) Copy your dtb

TX1

cp $DEVDIR/images/arch/arm64/boot/dts/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb $DEVDIR/kernel/dtb

TX2

cp $DEVDIR/images/arch/arm64/boot/dts/tegra186-quill-p3310-1000-c03-00-base.dtb $DEVDIR/kernel/dtb

10) Finally overwrite the default images with your own images to install and boot them using jetpack

cd $DEVDIR/images
cp -rf arch/arm64/boot/Image arch/arm64/boot/zImage packages/kernel_supplements.tbz2 $DEVDIR/kernel/
cp -rf packages/kernel_headers_custom.tbz2 $DEVDIR/kernel/kernel_headers.tbz2

11) Run the applybinaries script to generate the image to flash

cd $DEVDIR/
sudo ./apply_binaries.sh

As an important note, kernel will be located not in a partition but in /boot on the filesystem this make easier a kernel update.

Flashing the Tegra

A USB cable is required and must be connected to the recovery port prior to running the flashing command.

To flash the target Tegra device:

a. Put the target board into reset/recovery mode as follows:

Connect the Tegra to the Jetson board via CVM connector.
Power on the Jetson board.
Hold the recovery button and then press the reset button.
The Jetson board should be connected to the host computer via USB, so you can check if it is on reset/recovery mode by typing:

lsusb

You should see "NVidia Corp." as part of the returned message.

For more information you can check the User Guide: https://developer.download.nvidia.com/embedded/L4T/r27_Release_v1.0/Docs/Jetson_X2_Developer_Kit_User_Guide.pdf

b. On the host machine, copy the .dtb files located in the Linux_for_Tegra/images/arch/arm64/boot/dts directory to the Linux_for_Tegra/kernel/dtb/ directory.

c. Copy Image and zImage located in the Linux_for_Tegra/images/arch/arm64/boot/ directory to the Linux_for_Tegra/kernel directory.

d. Copy the modules from Linux_for_Tegra/images/modules/lib/modules/4.4.38/ to Linux_for_Tegra/rootfs/lib/modules/4.4.38

e. Run the following command from $DEVDIR":

If you are using microSD and eMMC:

sudo ./flash.sh -r -d kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb -K kernel/Image jetson-tx2 mmcblk1p1

And if you are using a only the eMMC:

sudo ./flash.sh -r -d kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb -K kernel/Image jetson-tx2 mmcblk0p1

Installing DTB

In previous versions of jetpack, updating the DTB was as easy as replacing the one in the boot folder of the boot directory and you could also just change the FDT entry in /boot/extlinux/extlinux.conf to use a different one. For Jetpack 3.1 this was changed and a separate partition is used to flash the DTB file and as per NVIDIA, you can only update it by flashing it again using the provided flash script.

Here is how to do it:

$DEVDIR/images/arch/arm64/boot/dts/*.dtb

TX1

Replace the dtb in $DEVDIR/kernel/dtb with yours (you might want to make a backup of the original DTB, just in case):

cp $DEVDIR/images/arch/arm64/boot/dts/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb $DEVDIR/kernel/dtb

Put the board into recovery mode and flash the DTB:

cd $DEVDIR
sudo ./flash.sh -r -k DTB jetson-tx1 mmcblk1p1

TX2

Replace the dtb in $DEVDIR/kernel/dtb with yours (you might want to make a backup of the original DTB, just in case):

cp $DEVDIR/images/arch/arm64/boot/dts/tegra186-quill-p3310-1000-c03-00-base.dtb $DEVDIR/kernel/dtb

Put the board into recovery mode and flash the DTB:

cd $DEVDIR
sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk1p1

NOTE: Use mmcblk1p1 for SDcard and mmcblk0p1 for emmc.

Flash DTB from the Jetson device itself

You'll need to generate signed/encrypted dtb files through the flash scripts and then flash it to the kernel-dtb partition of the Jetson device.

1. To generate the signed/encrypted dtb file follow these steps (tested JP 4.2 TX2):

cd $DEVDIR
sudo ./flash.sh --no-flash -r -k kernel-dtb jetson-tx2 mmcblk0p1

The above command will generate the following encrypted dtb file:

$DEVDIR/bootloader/tegra186-quill-p3310-1000-c03-00-base_sigheader.dtb.encrypt

2. Check the partition labels of your Jetson device:

nvidia@nvidia-desktop:~$ ls -al /dev/disk/by-partlabel
total 0
drwxr-xr-x 2 root root 660 jun 26 11:42 .
drwxr-xr-x 8 root root 160 jun 26 11:45 ..
lrwxrwxrwx 1 root root  16 jun 26 11:42 adsp-fw -> ../../mmcblk0p11
lrwxrwxrwx 1 root root  16 jun 26 11:42 adsp-fw_b -> ../../mmcblk0p12
lrwxrwxrwx 1 root root  15 jun 26 11:42 APP -> ../../mmcblk0p1
lrwxrwxrwx 1 root root  16 jun 26 11:42 BMP -> ../../mmcblk0p22
lrwxrwxrwx 1 root root  16 jun 26 11:42 BMP_b -> ../../mmcblk0p23
lrwxrwxrwx 1 root root  15 jun 26 11:42 bootloader-dtb -> ../../mmcblk0p6
lrwxrwxrwx 1 root root  15 jun 26 11:42 bootloader-dtb_b -> ../../mmcblk0p7
lrwxrwxrwx 1 root root  16 jun 26 11:42 bpmp-fw -> ../../mmcblk0p13
lrwxrwxrwx 1 root root  16 jun 26 11:42 bpmp-fw_b -> ../../mmcblk0p14
lrwxrwxrwx 1 root root  16 jun 26 11:42 bpmp-fw-dtb -> ../../mmcblk0p15
lrwxrwxrwx 1 root root  16 jun 26 11:42 bpmp-fw-dtb_b -> ../../mmcblk0p16
lrwxrwxrwx 1 root root  16 jun 26 11:42 CAC -> ../../mmcblk0p30
lrwxrwxrwx 1 root root  15 jun 26 11:42 cpu-bootloader -> ../../mmcblk0p4
lrwxrwxrwx 1 root root  15 jun 26 11:42 cpu-bootloader_b -> ../../mmcblk0p5
lrwxrwxrwx 1 root root  16 jun 26 11:42 eks -> ../../mmcblk0p10
lrwxrwxrwx 1 root root  16 jun 26 11:42 FBNAME -> ../../mmcblk0p21
lrwxrwxrwx 1 root root  16 jun 26 11:42 kernel -> ../../mmcblk0p26
lrwxrwxrwx 1 root root  16 jun 26 11:42 kernel_b -> ../../mmcblk0p27
lrwxrwxrwx 1 root root  16 jun 26 11:42 kernel-dtb -> ../../mmcblk0p28
lrwxrwxrwx 1 root root  16 jun 26 11:42 kernel-dtb_b -> ../../mmcblk0p29
lrwxrwxrwx 1 root root  15 jun 26 11:42 mts-bootpack -> ../../mmcblk0p2
lrwxrwxrwx 1 root root  15 jun 26 11:42 mts-bootpack_b -> ../../mmcblk0p3
lrwxrwxrwx 1 root root  16 jun 26 11:42 sc7 -> ../../mmcblk0p19
lrwxrwxrwx 1 root root  16 jun 26 11:42 sc7_b -> ../../mmcblk0p20
lrwxrwxrwx 1 root root  16 jun 26 11:42 sce-fw -> ../../mmcblk0p17
lrwxrwxrwx 1 root root  16 jun 26 11:42 sce-fw_b -> ../../mmcblk0p18
lrwxrwxrwx 1 root root  15 jun 26 11:42 secure-os -> ../../mmcblk0p8
lrwxrwxrwx 1 root root  15 jun 26 11:42 secure-os_b -> ../../mmcblk0p9
lrwxrwxrwx 1 root root  16 jun 26 11:42 SOS -> ../../mmcblk0p24
lrwxrwxrwx 1 root root  16 jun 26 11:42 SOS_b -> ../../mmcblk0p25
lrwxrwxrwx 1 root root  16 jun 26 11:42 UDA -> ../../mmcblk0p31

The partition of interest is the following:

kernel-dtb -> ../../mmcblk0p28

3. Flash the encrypted dtb file into the kernel-dtb Jetson device partition: Transfer the encrypted dtb file to your jetson device.

scp $DEVDIR/bootloader/tegra186-quill-p3310-1000-c03-00-base_sigheader.dtb.encrypt nvidia@10.251.101.96:

Overwrite dtb partition, and perform warm-reboot to take effect:

sudo dd if=tegra186-quill-p3310-1000-c03-00-base_sigheader.dtb.encrypt of=/dev/mmcblk0p28 bs=1M
sudo reboot

Bootloader

The bootloader is a combination of NVIDIA T-Boot (nvtboot) and U-Boot. You can read about the BOOT FLOW in the nvtboot wiki page.

According to the devtalk forum there are two ways to boot the board: fast boot and uboot

Fastboot is simpler, and offers no options during boot. It strictly looks for a kernel in the GPT partition from the install's "-k 6" option. This would probably be the way to go for a non-interactive embedded appliance (if not bare metal booting).

U-boot offers more options, and looks for a zImage in the /boot partition. Boot configuration can also be edited here, for example I threw in a couple of zImage variants (like debugging version and network options added), and edited extlinux.conf...and magically I can boot to any of those kernels at the serial console boot prompt (provided my eye is fast). No flash was required for any kernel beyond the first kernel.

Inside of the bootloader directory you will find several binaries or components, to get a description of these please read the nvtboot wiki page.

Build DTC

U-boot requires the Device Tree Compiler (dtc) which is used to compile device tree files contained in the U-Boot source tree. One version is included inside of the kernel. However, that binary is too old according to u-boot when trying to use that one

*** Your dtc is too old, please upgrade to dtc 1.4 or newer
$DEVDIR/images/dtc -v
Version: DTC 1.2.0-g37c0b6a0

In order to build it please run:

cd $DEVDIR/sources
git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git
cd dtc
make
cp dtc $DEVDIR/images/

Build u-boot

1. Backup original uboot files. You see which platform you are using checking the file in $DEVDIR/jetson-tx1.conf, it says:

SYSBOOTFILE=p2371-2180-devkit/extlinux.conf;

copy the files to the packages-backup directory

mkdir $DEVDIR/images/packages-backup/bootloader/
cp -rf $DEVDIR/bootloader/t210ref/p2371-2180-devkit $DEVDIR/images/packages-backup/bootloader/
cd $DEVDIR/bootloader/
cp nvtboot.bin nvtboot_cpu.bin u-boot-dtb.bin bpmp.bin warmboot.bin tos.img tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb $DEVDIR/images/packages-backup/bootloader/
cp $DEVDIR/bootloader/t210ref/p2371-2180 $DEVDIR/images/packages-backup/bootloader/

Check that all these components are flashed in one partition according to Jetpack output when flashing Tegra X1. The description of each of these components in on the t210-nvtboot-flow and tegra-boot-flow pages.

2. Go to uboot and set environment variables. You need to export the images directory path in order to allow u-boot find the DTC

cd $DEVDIR/sources/u-boot/
PATH=$DEVDIR/images:$PATH
export CROSS_COMPILE=/opt/linaro/gcc-linaro-5.3-2016.02-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
export ARCH=arm64

3. Clean and compile

make distclean
make p2371-2180_defconfig
make

4. Save your new binaries

mkdir $DEVDIR/images/bootloader
cp $DEVDIR/sources/u-boot/u-boot{,.bin,.dtb,-dtb.bin} $DEVDIR/images/bootloader

Note: L4T U-Boot does not use the kernel partition. The kernel is installed into the filesystem alongside the boot configuration file in /boot. Aside from this difference, U-Boot has the same internal eMMC partition layout as that used by cboot.

5. Now, copy your u-boot binaries to the directory used by Jetpack to look for them. As you can see in the DTB it will look for them in

target_board="t210ref";
BOOTLOADER="bootloader/${target_board}/p2371-2180/u-boot-dtb.bin";

To copy the files run

cp  $DEVDIR/images/bootloader/u-boot{,.bin,.dtb,-dtb.bin} $DEVDIR/bootloader/t210ref/p2371-2180/

6. Now run jetpack or follow the steps below to use flash.sh and let it install the new binaries as we did with the kernel. You can check the serial console output to see that u-boot has a newer date

U-Boot 2015.07-rc2 (Apr 28 2016 - 19:24:09 -0600)

If you are attached to the serial port while flashing you should see the serial port output log while flashing

Boot sequence

According to the U-boot guide provided by NVIDIA uboot scans booteable devices as follows:

External SD Card
Internal eMMC
USB Device
NFS Device

It looks for an extlinux.conf configuration file in the following directory of the bootable device:

<rootfs>/boot/extlinux

Upon finding the extlinux.conf file, U-Boot does the following.

Uses the sysboot command to read out boot configuration from extlinux.conf,
Loads kernel Image file and device tree file (/boot), and then
Boots the kernel.

extlinux.conf is a standard text-format sysboot configuration file that contains all boot information, it is installed in /boot according to the boot method.

Boot log and uboot environment

Following boot log was obtained when connected to the serial console using the default uboot environment.

The environment is defined in extlinux.conf, however, the environment variables can be set in the uboot code directly or directly when the board is booting using the typical uboot commands like: print, saveenv, printenv, etc

Filesystem

Jetpack comes with a sample filesystem with X and ubuntu and Canonical keeps updating the packages available, you can also use your own filesystem. In both cases you have to install on it the NVIDIA binaries, the instructions to do that are in the file Start_L4T_Docs.html, in the Getting started section. It mentions:

Extract the sample file system to the rootfs directory with this command:

$ sudo tar jxpf ../../Tegra-Linux-Sample-Root-Filesystem_<release_type>.tbz2

Run the apply_binaries.sh script to copy the NVIDIA user space libraries into the target file system:

$ cd ..
$ sudo ./apply_binaries.sh

If you are using a different rootfs, or if you have already configured your rootfs, apply the NVIDIA user space libraries by setting the LDK_ROOTFS_DIR environment variable to point to your rootfs. Then run the script, as shown above, to copy the binaries into your target file system.

If the apply_binaries.sh script installs the binaries correctly, the last message output from the script is “Success!”.

You can create your own ubuntu filesystem using .deb packages as sources following this instructions. The sample filesystem provided is a 32 bits filesystem, however, the kernel is a 64bits kernel. According to devtalk nvidia will be releasing a 64bits filesystem.

The remaining steps on this guide assumes that you are using the default filesytem or that you already have one built, specially for the flashing section, where flashing assumes that the filesystem already exists.

To receive notifications of new packages

1. Locate and edit the following file:

/etc/apt/sources.list

2. Add the following line:

deb http://ports.ubuntu.com/ubuntu-ports <distribution>-updates main universe

Where <distribution> is the name of the Ubuntu distribution your rootfs is based on. For example, for a rootfs based on the Trusty Tahr distribution of Ubuntu, add the line:

deb http://ports.ubuntu.com/ubuntu-ports trusty-updates main universe

Prerequisite

You have attached an Ethernet cable to the device through either the Ethernet port (if available) or through the USB Ethernet adapter. To install more packages

Boot the target device.
Verify your Ethernet connection.
Update the package list by executing:

$ sudo apt-get update

Note: Ensure that you run sudo apt-get update and not apt-get upgrade, which upgrades already installed packages. Do not confuse the two commands.

Install packages using apt-get. For example, to install wget execute this command:

$ sudo apt-get install wget

Flashing the board

NVIDIA provides several scripts to help to generate the images and to flash the resulting images. Including if you are going to install uboot or fastboot. First the board needs to be in recovery mode and then the main script to flash it is $DEVDIR/Linux_for_Tegra_tx1/flash.sh. What this script will do is to install or flash the image in the desired destination. It is important that the script is controlled by variables defined in the DTB file.

# Examples:
# ./flash.sh <target_board> mmcblk0p1			- boot <target_board> from eMMC
# ./flash.sh <target_board> mmcblk1p1			- boot <target_board> from SDCARD
# ./flash.sh <target_board> sda1			- boot <target_board> from USB device
# ./flash.sh -N <IPaddr>:/nfsroot <target_board> eth0	- boot <target_board> from NFS
# ./flash.sh -k LNX <target_board> mmcblk1p1		- update <target_board> kernel
# ./flash.sh -k EBT <target_board> mmcblk1p1		- update <target_board> bootloader
#
# Optional Environment Variables:
# BCTFILE ---------------- Boot control table configuration file to be used.
# BOARDID ---------------- Pass boardid to override EEPROM value
# BOOTLOADER ------------- Bootloader binary to be flashed
# BOOTPARTLIMIT ---------- GPT data limit. (== Max BCT size + PPT size)
# BOOTPARTSIZE ----------- Total eMMC HW boot partition size.
# CFGFILE ---------------- Partition table configuration file to be used.
# CMDLINE ---------------- Target cmdline. See help for more information.
# DEVSECTSIZE ------------ Device Sector size. (default = 512Byte).
# DTBFILE ---------------- Device Tree file to be used.
# EMMCSIZE --------------- Size of target device eMMC (boot0+boot1+user).
# FLASHAPP --------------- Flash application running in host machine.
# FLASHER ---------------- Flash server running in target machine.
# IGNOREFASTBOOTCMDLINE -- Block fastboot from filling unspecified kernel
#                          cmdline parameters with its defaults.
# INITRD ----------------- Initrd image file to be flashed.
# ITSFILE ---------------- Multi image u-boot package template file.
# KERNEL_IMAGE ----------- Linux kernel zImage file to be flashed.
# MTS -------------------- MTS file name such as mts_si.
# MTSPREBOOT ------------- MTS preboot file name such as mts_preboot_si.
# NFSARGS ---------------- Static Network assignments.
#			   <C-ipa>:<S-ipa>:<G-ipa>:<netmask>
# NFSROOT ---------------- NFSROOT i.e. <my IP addr>:/exported/rootfs_dir.
# ODMDATA ---------------- Odmdata to be used.
# ROOTFSSIZE ------------- Linux RootFS size (internal emmc/nand only).
# ROOTFS_DIR ------------- Linux RootFS directory name.
# TEGRABOOT -------------- lowerlayer bootloader such as nvtboot.bin.
# UBOOTSCRIPT ------------ U-boot HUSH boot script file.
# UBOOT_TEXT_BASE -------- U-boot Image Load Address.
# UIMAGE_LABEL ----------- Kernel version for U-boot image header.
# UIMAGE_NAME ------------ uImage file name.
# WB0BOOT ---------------- Warmboot code such as nvtbootwb0.bin

Recovery mode

To put the board into force USB Recovery Mode:

1. Power down the device. If connected, remove the AC adapter from the device. The device must be
powered OFF, and not in a suspend or sleep state.
2. Connect the Micro-B plug on the USB cable to the Recovery (USB Micro-B) Port on the device and
the other end to an available USB port on the host PC.
3. Connect the power adapter to the device.
4. Press POWER button
5. Press and hold the RECOVERY FORCE (REC) button.
6. While pressing the RECOVERY FORCE button, press and release the RESET button.
7. Wait 2 seconds and release the RECOVERY FORCE button

All remaining examples assume that the board is in recovery mode

Flash filesystem

Emmc

To flash the filesystem in the emmc you need to run:

TX1

cd $DEVDIR/
sudo ./flash.sh jetson-tx1 mmcblk0p1

TX2

cd $DEVDIR/
sudo ./flash.sh jetson-tx2 mmcblk0p1

In the TX2 board, to flash the kernel and the DTB in the EMMC, you also have to flash the File System. In order to pass specific binaries you can use -d and -K options. Also, if you want to use an existent system.img and avoid creating a new one you can use -r option. For example:

sudo ./flash.sh -r -d kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb -K kernel/Image jetson-tx2 mmcblk0p1

USB drive

Important: Only for USB drive does tegra need firmware loaded from the filesystem so the USB port is not enabled in uboot and then it causes problems to mount the fs from there. You can read about it here.

To flash the filesystem to a USB drive you need to put the filesystem on the USB drive first.

1) Insert your USB flash/drive in your PC and check where it was mounted

mount
/dev/sdb1 on /media/dsoto/EF02-5626 type vfat (rw,nosuid,nodev,uid=1000,gid=1000,shortname=mixed,dmask=0077,utf8=1,showexec,flush,uhelper=udisks2)

2) Format the USB drive in your PCB

umount /dev/sdb1
sudo mkfs.ext4 /dev/sdb1

3) Copy the filesystem to the USB drive. It takes a while, its size is normally 2.4GB

mkdir $DEVDIR/images/fs
sudo mount /dev/sdb1 $DEVDIR/images/fs 
cd $DEVDIR/rootfs/
sudo cp -a * $DEVDIR/images/fs  && sync

4) Unmount the USB drive and remove it

sudo umount $DEVDIR/images/fs

5) Finally connect the USB drive to the Jetson board put the Jetson on recovery mode, flash the board to look for the filesystem on the USB port:

TX1

cd $DEVDIR/
sudo ./flash.sh jetson-tx1 sda1 #it can be other mount point

TX2

cd $DEVDIR/
sudo ./flash.sh jetson-tx2 sda1 #it can be other mount point

SD card

Similar to USB but you need to replace the extlinux.conf so before copying the filesystem into the SD card run:

TX1

sudo cp $DEVDIR/bootloader/t210ref/p2371-2180-devkit/extlinux.conf.sdcard $DEVDIR/rootfs/boot/extlinux/extlinux.conf

TX2

sudo cp $DEVDIR/bootloader/t186ref/p2771-0000/extlinux.conf.sdcard $DEVDIR/rootfs/boot/extlinux/extlinux.conf

then flash it

cd $DEVDIR/

TX1

sudo ./flash.sh jetson-tx1 mmcblk1p1

TX2

sudo ./flash.sh jetson-tx2 mmcblk1p1

NFS

1) First you need to set up your NFS server using these instructions.

2) Open file in $DEVDIR/rootfs/etc/network/interfaces and add

auto eth0
iface eth0 inet manual

3) Then flash the board to generate and install the initrd image to use nfs, you need to do this everytime that you recompile the kernel

cd $DEVDIR/
MY_IPADDRESS=192.169.0.3
sudo ./flash.sh -N $MY_IPADDRESS:$DEVDIR/rootfs jetson-tx1 eth0 #use jetson-tx2 for TX2

Flash kernel

To flash your kernel you just need to copy it to /boot in your filesystem. The old method to flash it in the LNX partition (./flash.sh -k LNX <target_board> mmcblk1p1) is NOT longer used

sudo cp $DEVDIR/images/zImage $DEVDIR/images/Image $DEVDIR/rootfs/boot/

if you are not booting from NFS then you need to reflash your filesysm in order to update the kernel. However, likely you did some changes in the kernel modules so it is recommended to run the apply_binaries.sh script instead of only copying the kernel images.

cd $DEVDIR/
sudo ./apply_binaries.sh

then flash your filesystem again.

Option 2.

If you don't want to reflash the complete file system, you can just move the Image to your board using SCP (your board and your host computer have to be in the same network).

1) Find your board's IP address:

From your board run:

ifconfig

you'll see something like:

eth0      Link encap:Ethernet  HWaddr 0c:54:a5:1d:18:fe  
          inet addr:10.251.101.40  Bcast:10.251.101.255  Mask:255.255.255.0

So in this case the IP address is 10.251.101.40

2) From your host computer copy the Image to your board.

BOARD_IP_ADDRESS=10.251.101.40
scp $DEVDIR/images/arch/arm64/boot/Image nvidia@BOARD_IP_ADDRESS:/tmp

You'll be asked for a password, use nvidia

3) From your board, copy the kernel Image to the /boot folder:

sudo cp /tmp/Image /boot

You'll be asked for a password again, use ubuntu if your user is ubuntu or nvidia if your user is nvidia.

4) Reboot the board.

sudo reboot

At this point you should be using the new kernel Image, you can verify with the uname command:

uname -a

Just make sure the date is correct: Linux tegra-ubuntu 4.4.38+ #35 SMP PREEMPT Tue Nov 28 12:29:01 CST 2017 aarch64 aarch64 aarch64 GNU/Linux

In the TX2 with Jetpack 3.2.1, you can use below commands to flash kernel and DTB independently:

Kernel

sudo ./flash.sh -r -k kernel -K kernel/Image jetson-tx2 mmcblk0p1

DTB

sudo ./flash.sh -r -k kernel-dtb -d kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb jetson-tx2 mmcblk0p1

Flash bootloader

Update the content of the EBT partition where uboot is located

cd $DEVDIR/64_TX1/Linux_for_Tegra_64_tx1/
sudo ./flash.sh -k EBT jetson-tx1 mmcblk1p1

Serial Console

J21 contains the Serial Console signals as can be seen in the pin out header. Basically:

J21 Pin 8 (UART 1 TXD) -> Cable RXD (Yellow)
J21 Pin 10 (UART 1 RXD) -> Cable TXD (Orange)
TX1 J21 Pin 9 (GND) -> Cable GND (Black)

The listed colors match the wire color if you are using an FTDI TTL-232R-3V3-AJ USB to serial cable .

There is a nice video with details.

References

https://devtalk.nvidia.com/default/topic/929186/jetson-tx1/jetson-tx1-kernel-compilation/
https://devtalk.nvidia.com/default/topic/762653/?comment=4654303
https://devtalk.nvidia.com/default/topic/901677/building-tx1-kernel-from-source/?offset=17
https://devtalk.nvidia.com/default/topic/760180/newbie-building-l4t-from-source-which-branch-tag-/
https://wiki.ubuntu.com/ARM/RootfsFromScratch
https://devtalk.nvidia.com/default/topic/929186/jetson-tx1/jetson-tx1-kernel-compilation/post/4854603/#4854603
https://tls.mbed.org/kb/development/arm-thumb-error-r7-cannot-be-used-in-asm-here
https://devtalk.nvidia.com/default/topic/914941/jetson-tx1/custom-kernel-compilations/post/4799773/#4799773
https://devtalk.nvidia.com/default/topic/914941/jetson-tx1/custom-kernel-compilations/2
https://devtalk.nvidia.com/default/topic/897280/jetson-tx1/jetson-tx1-with-l4t-23-1-doesn-t-support-native-aarch64-binaries-/post/4732697/#4732697
http://elinux.org/Jetson_TX1
https://devtalk.nvidia.com/default/topic/901677/jetson-tx1/building-tx1-kernel-from-source/post/4749509/#4749509
https://devtalk.nvidia.com/default/topic/905345/jetson-tx1/built-tx1-u-boot-from-source/
http://http.download.nvidia.com/tegra-public-appnotes/tegra-boot-flow.html
https://devtalk.nvidia.com/default/topic/820035/?comment=4494578
http://http.download.nvidia.com/tegra-public-appnotes/index.html
https://devtalk.nvidia.com/default/topic/781056/jetson-tk1-boot-from-nfs-system-hangs-up-during-services-boot/?offset=3
https://devtalk.nvidia.com/default/topic/744908/jetson-tk1/jetson-tk1-boot-over-nfs/