View source for Raspberry Pi HQ camera IMX477 Linux driver for Jetson

{{#seo:
|title=Raspberry Pi HQ camera IMX477 Linux driver for Jetson
|title_mode=replace
|description={{{description|Read RidgeRun’s detailed guide about the Raspberry Pi HQ camera Sony IMX477 Linux driver for Jetson.}}}
}}
{|
| style="text-align:right; vertical-align:top;" | __TOC__
| {{Mpo-shoppingcart & NVIDIA logo & RR Contact}}
|}
{|
| {{GStreamer debug}}
|}
{{Driver List Information}}

==Introduction to Sony IMX477 Linux driver==

[https://www.ridgerun.com/ RidgeRun] is actively working to enable customers with great software solutions on top of powerful hardware. For this reason, RidgeRun is currently working to create the V4L2 driver for the Raspberry Pi HQ camera. This wiki contains a brief introduction to the sensor and hardware that will be used to develop the driver. 

The instructions in this wiki are useful for Jetson Xavier NX and Jetson Nano, keep in mind that for Jetson Orin with a J20 adapter, you can follow the following instructions [[NVIDIA_Jetson_Orin/Video_Capture_and_Display/Cameras/IMX477_J20 | here]]. Also please keep in mind that the driver can be ported to other Jetson platforms if needed. Please [https://www.ridgerun.com/contact contact us] if you require more information.

==Raspberry Pi HQ camera==
<br>
[[Image:rpi-hq-imx477.png|thumb|center|500px]]

===Camera Features===

* 12.3-Megapixel high-resolution Sony IMX477 sensor.
* 1.55μm × 1.55μm Pixel Size – double the pixel area of IMX219 for improved low-light performance.
* Back-illuminated sensor architecture for improved sensitivity.
* Support for different C- and CS-mount lenses.
* Integrated back-focus adjustment ring and tripod mount.

[[Image:rpi-hq-imx477-sensor.jpg|thumb|center|500px]]

===Sony IMX477 Basic Drive Mode===

The Raspberry Pi HQ camera uses the IMX477 CMOS sensor of Sony to capture. Such a sensor provides the modes included in the table below. At this point, RidgeRun is working to support one mode only but others can be added per request.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;
|-
! '''Drive Mode''' !! '''Number of active pixels''' !! '''Maximum frame rate<br>[frame/s]''' !! '''Output Interface''' !! '''ADC<br>[bit]'''
|-
| rowspan="2" | '''Full (4:3) (Normal)''' || rowspan="2" | 4056 x 3040<br>approx. 12.33 M pixels
| 60 || CSI-2 || 10
|-
| 40 || CSI-2 || 12
|-
| '''Full (4:3) (DOL-HDR)''' || 4056 x 3040<br>approx. 12.33 M pixels || DOL 2 frame:30<br>DOL 3 frame:15 || CSI-2 || 10
|-
| '''Full (16:9) 4K2K (Normal)''' || 4056 x 2288<br>approx. 9.28 M pixels || 79 || CSI-2 || 10
|-
| '''Full (16:9) 4K2K (DOL-HDR)''' || 4056 x 2288<br>approx. 9.28 M pixels || DOL 2 frame:39<br>DOL 3 frame:19 || CSI-2 || 10
|-
| '''Full (4:3) Binning (Normal)''' || 2028 x 1520<br>approx. 3.08 M pixels || 179 || CSI-2 || 10
|-
| '''Full (16:9) Binning 1080P (Normal)''' || 2028 x 1128<br>approx. 2.29 M pixels || 240 || CSI-2 || 10
|-
| '''Full (16:9) Binning 720P (Normal)''' || 1348 x 750<br>approx. 1.01 M pixels || 240 || CSI-2 || 10
|-
| '''Full (16:9) Scaling 1080P (Normal)''' || 2024 x 1142<br>approx. 2.31 M pixels || 79 || CSI-2 || 10
|-
| '''Full (16:9) Scalling 720P (Normal)''' || 1348 x 762<br>approx. 1.03 M pixels || 79 || CSI-2 || 10
|-
|}

===Compatibility with NVIDIA<sup>®</sup>Jetson™ Platforms===

The Raspberry Pi HQ camera module requires a hardware modification in order to work with Jetson Platforms.
Jetson Platforms provide 1.8V for reset GPIO in the camera interface, but the camera module requires 3.3V.

To fix this issue a resistor labeled as R8 must be removed from the camera module. If you decide to apply this fix it is under your own risk. RidgeRun is not responsible for any damage caused to your board. The following picture shows the location of the R8 in the module.
<br>
<br>
[[Image:R8ToBeRemoved.png|thumb|center|500px]] 
<div style="text-align: center;">'''Figure 1'''. R8 resistor on RPi HQ camera module.</div>

==IMX477 Linux driver for NVIDIA<sup>®</sup>Jetson™==

RidgeRun Engineering has developed the driver with NVIDIA Corporation and Leopard Imaging Inc. as a collaborative initiative.
In this section, you will find the instructions to patch the JetPack sources enabling the IMX477 sensor driver for the Jetson Xavier NX and Nano. These  instructions have been validated in the following JetPack versions:
* Jetpack 4.4
* Jetpack 4.4.1
* Jetpack 4.5

For '''Jetson Orin,''' a port for JetPack 5.0.1 was made, you can find all the information [[NVIDIA_Jetson_Orin/Video_Capture_and_Display/Cameras/IMX477_J20 | here]]

===Download JetPack===

The current version of the driver is supported in Jetpack 4.4, 4.4.1, and 4.5. Porting the driver to other versions and platforms is possible.

A port for is available for the '''Jetson Orin''' with J20 adapter in [[NVIDIA_Jetson_Orin/Video_Capture_and_Display/Cameras/IMX477_J20 | here]]

1. Download and install the [https://developer.nvidia.com/nvidia-sdk-manager NVIDIA SDK Manager] (membership is required).

2. Put your NVIDIA membership credentials in the SDK Manager main window:
[[Image:SDKMCredentials.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 2'''. Required SDK Manager credentials.</div>
3. Select JetPack version and Jetson [Xavier NX | Nano] as shown below and press ''Continue'':
[[Image:SDKMOptions2.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 3'''. Required SDK Manager options.</div>
4. Accept the terms and conditions and press ''Continue''. The SDK Manager will download and install the selected components:
[[Image:SDKMInstall.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 4'''. SDK Manager download and install software.</div>
5. Select '''Manual Setup'''. Make sure the board is in ''Recovery Mode'' and then press '''Flash''': 

[[Image:ManualOption.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 5'''. Manual setup to start the flashing process.</div>

====Recovery Mode for Jetson NX====

{{Ambox
|type=notice
|small=left
|issue=To set the Xavier NX developer kit in '''Recovery Mode''' the following steps are required:
|style=width:unset;
}}

# Ensure the device is powered off and the power adapter is disconnected.
# Verify that a microSD card is inserted in the Jetson Xavier NX module's card slot.
# Place a jumper across the Force Recovery Mode pins (9 and 10) on the button header [J14]. Please, see Figure 6 below.
# Connect your host computer to the developer kit's USB Micro-B connector.
# Connect the power supply to the power jack [J16]. The developer kit automatically powers on in Force Recovery Mode.
# Remove the jumper from the Force Recovery Mode pins.
# Use the <code>lsusb</code> command to check if the <code>NVidia corp</code> message is shown.<br>
<br>
[[Image:recmodeNX.png|thumb|center|650px]]
<div style="text-align: center;">'''Figure 6'''. Jumper between pins 9 and 10 to get the board on Recovery Mode (top view).</div><br>

====Recovery Mode for Jetson Nano====
* To set the Jetson Nano in ''Recovery Mode'', follow these steps:
# Ensure the device is powered off and the power adapter is disconnected.
# Verify that a microSD card is inserted in the Jetson Nano SD card slot.
# Place a jumper across the Force Recovery pins.
# Enable the use of the Power Jack [J15] by placing a jumper across the Power Select Header [J48] pins.
# Connect your host computer to the device's Micro USB connector.
# Connect the power supply to the power jack [J25]. The developer kit automatically powers on in Force Recovery Mode.
# Remove the jumper from the Force Recovery Mode pins.

{{Ambox
|type=notice
|small=left
|issue=
 *For the B01 carrier board, the force recovery pins are the 9 and 10 pins (FC REC and GND) of the Button Header. The B01 Button Header is located on the edge of the carrier board under the heat sink and SD card slot.<br>
 *For the A02 carrier board, the force recovery pins are the 3 and 4 pins of the Button Header.
|style=width:unset;
}}
<br>

6. When the OS has been installed successfully in the [Xavier NX | Nano], you will be asked to enter the board IP, a username and a password, as shown below in Figure7: 

[[Image:OSIPOption.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 7'''. Required username and password for your profile in the board.</div>

7. Don't close the window of Figure 7 and don't modify its content yet. You need to connect a screen to the board via HDMI, a keyboard/mouse via USB port, and a network cable via Ethernet port. You can remove the jumper that keeps the board in ''Recovery Mode'' and then reboot the Xavier board at this point if necessary.

8. After reboot, you will be asked to accept the terms of the license' on the board's screen. Select the checkbox and press ''Continue''.

9. Select your language in the next screen and press ''Continue'':

[[Image:languageNX.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 8'''. Language selection.</div>

10. Select your keyboard layout and press ''Continue'':

[[Image:keyboardNX.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 9'''. Keyboard layout selection.</div>

11. Select your location and press ''Continue'':

[[Image:locationNX.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 10'''. Location selection.</div>

12. Enter a username and a password. Then, press ''Continue'':

[[Image:usernameNX.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 11'''. User credentials for Ubuntu.</div>

13. Wait for the system to apply the changes. The board will restart automatically.

14. Now you are able to access Ubuntu as a new user in the [Xavier NX | Nano]. You can find the board IP by using <code>ifconfig</code> from a terminal.

15. Get back to the window presented in Figure 7 (in your host PC) and enter the IP, username, and password that the [Xavier NX | Nano] has been configured with. Then press ''Install''.

[[Image:preinstallNX.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 12'''. User credentials to start the software installation process.</div>

16. The software installation process should finish successfully, as shown in Figure 13:

[[Image:finishNX.png|thumb|center|650px]] 
<div style="text-align: center;">'''Figure 13'''. Finishing software install.</div>

<br>
{{Ambox
|type=notice
|small=left
|issue='''IMPORTANT:'''
*From now on we will refer as <code>$JETPACK_DIR/</code> to the directory where the SDK Manager placed the <code>JetPack_4.4_Linux_JETSON_[PLATFORM]_DEVKIT</code> / <code>JetPack_4.4.1_Linux_JETSON_[PLATFORM]_DEVKIT</code> / <code>JetPack_4.5_Linux_JETSON_[PLATFORM]_DEVKIT</code>folder.<br>
      Where <code>[PLATFORM]</code> must be replaced by <code>XAVIER_NX</code> or <code>NANO</code>. This directory is usually <code>$HOME/nvidia/nvidia_sdk.<code>
|style=width:unset;
}}
<br>

== Installing the Driver - Option A: Debian Packages (Recommended) ==

This is the simplest way to install the IMX477 driver on Jetson platforms.

===Downloading the debian packages===

You can download the Debian packages according to your platform from the following link:

[https://drive.google.com/drive/folders/1E1bE57VlXqadLGg8sSw8KwW7AJRWjbmG?usp=sharing Kernel Debian Packages]

<br>
{{Ambox
|type=notice
|small=left
|issue='''Debian files for Jetpack 4.4.1 and 4.5:'''
For Jetpack 4.4.1 and 4.5 the IMX477 module already comes built-in with the default kernel Image provided by NVIDIA. Only the dtb needs to be changed to enable the IMX477, for this reason, we provide only the dtb Debian packages for Jetpack 4.4.1 and 4.5.
|style=width:unset;
}}
<br>

===Installing the debian packages===

====Jetson Xavier NX====

* Copy the kernel and dtb Debian packages to your board filesystem:

<pre>
scp debian_nx/nvidia-l4t-kernel_4.9.140-tegra-32.4.3-20200625213407_arm64.deb [jetson-username]@[jetson-ip]:/home/nvidia/
scp debian_nx/nvidia-l4t-kernel-dtbs_4.9.140-tegra-32.4.3-20200625213407_arm64.deb [jetson-username]@[jetson-ip]:/home/nvidia/
</pre>

* Install the debian packages in your board:
<pre>
sudo apt-get install --reinstall ./nvidia-l4t-kernel_4.9.140-tegra-32.4.3-20200625213407_arm64.deb
sudo apt-get install --reinstall ./nvidia-l4t-kernel-dtbs_4.9.140-tegra-32.4.3-20200625213407_arm64.deb
</pre>

* Enable the dtb changes by modifying the <code>/boot/extlinux/extlinux.conf</code> file:
<pre>
 TIMEOUT 30
 DEFAULT primary

 MENU TITLE L4T boot options

 LABEL primary
       MENU LABEL primary kernel
       LINUX /boot/Image
       INITRD /boot/initrd
       APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 
       FDT /boot/tegra194-p3668-all-p3509-0000.dtb
</pre>

* Reboot your board:
<pre>
sudo reboot
</pre>

====Jetson Nano====

* Copy the kernel and dtb debian packages to your board filesystem:

<pre>
scp debian_nano/nvidia-l4t-kernel_4.9.140-tegra-32.4.3-20200625213809_arm64.deb [jetson-username]@[jetson-ip]:/home/nvidia/
scp debian_nano/nvidia-l4t-kernel-dtbs_4.9.140-tegra-32.4.3-20200625213809_arm64.deb [jetson-username]@[jetson-ip]:/home/nvidia/
</pre>

* Install the debian packages in your board:
<pre>
sudo apt-get install --reinstall ./nvidia-l4t-kernel_4.9.140-tegra-32.4.3-20200625213809_arm64.deb
sudo apt-get install --reinstall ./nvidia-l4t-kernel-dtbs_4.9.140-tegra-32.4.3-20200625213809_arm64.deb
</pre>

* Enable the dtb changes by modifying the <code>/boot/extlinux/extlinux.conf</code> file:
<br>
{{Ambox
|type=notice
|small=left
|issue='''The FDT depends on the Nano revision:'''
'''For Jetson Nano B01:''' replace <dtb path> with /boot/tegra210-p3448-0000-p3449-0000-b00.dtb

'''For Jetson Nano A02:''' replace <dtb path> with /boot/tegra210-p3448-0000-p3449-0000-a02.dtb
|style=width:unset;
}}
<br>
<pre>
 TIMEOUT 30
 DEFAULT primary
 MENU TITLE L4T boot options
 LABEL primary
       MENU LABEL primary kernel
       LINUX /boot/Image
       INITRD /boot/initrd
       APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0
       FDT <dtb path>
</pre>

* Reboot your board:
<pre>
sudo reboot
</pre>

* After these steps, the driver should be working and you can run some capture tests described below.
* '''IMPORTANT''': You may see some washed-out colors in the captured image. This is because you haven't applied the ISP configuration file. Make sure to install the ISP file that you can find in the repo using [[Raspberry_Pi_HQ_camera_IMX477_Linux_driver_for_Jetson#ISP_camera_overrides_installation | these instructions]].

== Installing the Driver - Option B: JetPack sources patch==

===Download the JetPack sources===

The sources can be downloaded from NVIDIA's Download Center:

Jetpack 4.4
* [https://developer.nvidia.com/embedded/L4T/r32_Release_v4.3/sources/T210/public_sources.tbz2 Jetson Nano sources]
* [https://developer.nvidia.com/embedded/L4T/r32_Release_v4.3/sources/T186/public_sources.tbz2 Jetson Xavier NX sources]
Jetpack 4.4.1
* [https://developer.nvidia.com/embedded/L4T/r32_Release_v4.4/r32_Release_v4.4-GMC3/Sources/T210/public_sources.tbz2 Jetson Nano sources]
* [https://developer.nvidia.com/embedded/L4T/r32_Release_v4.4/r32_Release_v4.4-GMC3/Sources/T186/public_sources.tbz2 Jetson Xavier NX sources]
Jetpack 4.5
* [https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/sources/T210/public_sources.tbz2 Jetson Nano sources]
* [https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/sources/T186/public_sources.tbz2 Jetson Xavier NX sources]

1. You must be able to download a file called <code>public_sources.tbz2</code>.

2. Untar the sources
<pre>
tar -xvf public_sources.tbz2
</pre>

3. Let's unpack the <code>kernel_src.tbz2</code> and move the <code>hardware/</code> and <code>kernel/</code> folders into <code>Linux_for_Tegra/</code> inside the JetPack directory <code>$JETPACK_DIR/JetPack_[JETPACK_VERSION]_Linux_JETSON_[PLATFORM]_DEVKIT/</code>:

<pre>
cd Linux_for_Tegra/source/public/
tar -xvf kernel_src.tbz2
JETSON_KERNEL_SOURCE=$JETPACK_DIR/JetPack_[JETPACK_VERSION]_Linux_JETSON_[PLATFORM]_DEVKIT/Linux_for_Tegra/source/
mkdir -p $JETSON_KERNEL_SOURCE
mv hardware/ kernel/ $JETSON_KERNEL_SOURCE
</pre>

Where <code>[PLATFORM]</code> must be replaced by <code>XAVIER_NX</code> or <code>NANO</code>, and <code>[JETPACK_VERSION]</code> must be replaced by <code>4.4</code>, <code>4.4.1</code> or <code>4.5</code>.

===Patch instructions===

====Download the patches====

RidgeRun has created a [https://github.com/RidgeRun/NVIDIA-Jetson-IMX477-RPIV3.git repository] from where you can download the patch files.

<pre>
git clone https://github.com/RidgeRun/NVIDIA-Jetson-IMX477-RPIV3.git
</pre>

Older versions can be found in '''jetpack-4.4''' and '''jetpack-4.4.1''' branches in the repository. You just have to switch to the jetpack version branch you want to use.

Once you have downloaded the repository, move the files to the <code>$JETSON_KERNEL_SOURCE</code> directory (create the patches directory inside JETSON_KERNEL_SOURCE if necessary):

<pre>
KERNEL_PATCH=$(pwd)/NVIDIA-Jetson-IMX477-RPIV3/
mv -v $KERNEL_PATCH/patches_[PLATFORM]/* $JETSON_KERNEL_SOURCE/patches/
</pre>
Where <code>[PLATFORM]</code> must be replaced by <code>nx</code> or <code>nano</code>.

====Apply the patches====
<pre>
cd $JETSON_KERNEL_SOURCE/patches
quilt push
</pre>

===Kernel build instructions===

Once the sources have been patched, perform the following steps in order to build and install the driver.

====Install the Toolchain====

NVIDIA recommends using the Linaro 7.3.1 2018.05 toolchain. Download the pre-built toolchain binaries as follows:
<pre>
wget http://releases.linaro.org/components/toolchain/binaries/7.3-2018.05/aarch64-linux-gnu/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
</pre>

Execute the following commands to extract the toolchain:
<pre>
mkdir -p $HOME/l4t-gcc
cd $HOME/l4t-gcc
tar xf gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
</pre>

====Build the kernel====

* Create the main directories:
<pre>
cd $JETSON_KERNEL_SOURCE/../
mkdir -p modules/
mkdir -p packages/
mkdir -p dtb/
</pre>

* Establish the building paths:
<pre>
CROSS_COMPILE=$HOME/l4t-gcc/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
KERNEL_OUT=$JETSON_KERNEL_SOURCE/../build
KERNEL_MODULES_OUT=$JETSON_KERNEL_SOURCE/../modules
</pre>

* Use the following command to install dependencies on Ubuntu:
<pre>
sudo apt install libncurses-dev
</pre>

* Configure and build kernel:
<pre>
cd $JETSON_KERNEL_SOURCE
make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT tegra_defconfig
make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT menuconfig
</pre>

* Select:
<pre>
Device Drivers  --->
  <*> Multimedia support  --->
      NVIDIA overlay Encoders, decoders, sensors and other helper chips  --->
          <*> IMX477 camera sensor support
</pre>
Press the ''Y'' key in order to select the IMX477 option.
Go back by hitting the double ''Esc'' key until you get the message: ''Do you want to save your new configuration?'', select ''Yes'' and press ''Enter''.

* Compile kernel, device tree, and modules:

<pre>
make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j4 Image
make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j4 dtbs
make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j4 modules
make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT modules_install INSTALL_MOD_PATH=$KERNEL_MODULES_OUT
</pre>

===Flash the Jetson===

====Flash the Xavier NX====

Please, make sure the board is in [[Raspberry_Pi_HQ_camera_IMX477_Linux_driver_for_Jetson#Recovery_Mode_for_Jetson_NX|Recovery Mode]].

* Copy the generated kernel and dtb:

<pre>
cd $JETSON_KERNEL_SOURCE/../
cp build/arch/arm64/boot/dts/tegra194-p3668-all-p3509-0000.dtb ./kernel/dtb
cp build/arch/arm64/boot/Image ./kernel
</pre>

* Flash the dtb:
<pre>
sudo ./flash.sh -r -k kernel-dtb -d kernel/dtb/tegra194-p3668-all-p3509-0000.dtb jetson-xavier-nx-devkit mmcblk0p1
</pre>

* Copy the kernel image:
<pre>
scp ./kernel/Image <nvidia-nx-user>@<nvidia-nx-ip>:/tmp/
</pre>

* Move the Image from <code>/tmp</code> to <code>/boot</code> inside the Xavier NX's filesystem:
<pre>
sudo mv /tmp/Image /boot/
</pre>

====Flash the Nano====
<br>
{{Ambox
|type=notice
|small=left
|issue='''The DTB depends on the Nano revision:'''
'''For Jetson Nano B01:''' replace <dtb> with tegra210-p3448-0000-p3449-0000-b00.dtb

'''For Jetson Nano A02:''' replace <dtb> with tegra210-p3448-0000-p3449-0000-a02.dtb
|style=width:unset;
}}
<br>
* Make sure your extlinux.conf file at '''/boot/extlinux/extlinux.conf''' in your Nano's filesystem looks as follows by adding the FDT line (remember to change the <dtb> depending on your board).

<pre>
TIMEOUT 30
DEFAULT primary

MENU TITLE L4T boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      FDT /boot/<dtb>
      APPEND ${cbootargs} quiet
</pre>

* Copy the kernel image and DTB:

<pre>
cd $JETSON_KERNEL_SOURCE/../
scp build/arch/arm64/boot/Image <nvidia-nano-user>@<nvidia-nano-ip>:/tmp/
scp build/arch/arm64/boot/dts/<dtb> <nvidia-nano-user>@<nvidia-nano-ip>:/tmp/
</pre>

* Move the files from <code>/tmp</code> to <code>/boot</code> inside the Nano's filesystem:

<pre>
sudo mv /tmp/Image /boot/
sudo mv /tmp/<dtb> /boot/
</pre>

===Copy the kernel modules===

* You can just access your SD card in your PC and copy the '''KERNEL_MODULES=$JETPACK_DIR/JetPack_[JETPACK_VERSION]_Linux_JETSON_[PLATFORM]_DEVKIT/Linux_for_Tegra/modules/lib/modules/4.9.201/*''' directory content into '''/lib/modules/4.9.201/''' in your SD card to update the modules. Where <code>[PLATFORM]</code> must be replaced by <code>XAVIER_NX</code> or <code>NANO</code>, and <code>[JETPACK_VERSION]</code> must be replaced by <code>4.4</code>, <code>4.4.1</code> or <code>4.5</code>.

* Or you can update with your compiled modules  If your board is up you can update the kernel modules as follows:

<pre>
KERNEL_MODULES=$JETPACK_DIR/JetPack_[JETPACK_VERSION]_Linux_JETSON_[PLATFORM]_DEVKIT/Linux_for_Tegra/modules/lib/modules/4.9.140
cd $KERNEL_MODULES
# use rsync instead of scp to avoid symlink recursion problems
rsync --progress -avhe ssh ../4.9.201 <nvidia-nx-user>@<nvidia-nx-ip>:/tmp/
</pre>

Where <code>[PLATFORM]</code> must be replaced by <code>XAVIER_NX</code> or <code>NANO</code>, and <code>[JETPACK_VERSION]</code> must be replaced by <code>4.4</code>, <code>4.4.1</code> or <code>4.5</code>.


* Move the modules from <code>/tmp</code> to the <code>/lib/modules/</code> folder inside the Jetson's filesystem and reboot:

<pre>
sudo mv /tmp/4.9.201/ /lib/modules/
sudo reboot
</pre>

* If you find that the directory name is '''/lib/modules/4.9.201-tegra/''', change the directory name to '''/lib/modules/4.9.201/'''
* After these steps, the driver should be working and you can run some capture tests described below.
* '''IMPORTANT''': You may see some washed-out colors in the captured image. This is because you haven't applied the ISP configuration file. Make sure to install the ISP file that you can find in the repo using [[Raspberry_Pi_HQ_camera_IMX477_Linux_driver_for_Jetson#ISP_camera_overrides_installation | these instructions]].

==ISP camera overrides installation==

Please, visit the [[JetsonTX2/HowTo%27s/NVIDIA_Jetson_ISP_Control#Custom_ISP_Configuration | Custom ISP Configuration page]] in order to learn how to use the <code>camera_overrides.isp</code> file to configure the ISP.

==Supported Features==

===Resolutions and framerates===

* 1920x1080 @ 60fps
* 4032x3040 @ 30fps

===Controls===

* Gain
* Exposure
* Framerate
* Group Hold

==Example Pipelines==

Find some example pipelines to use the IMX477 on Jetson Xavier NX below:

===Display=== 

====1920x1080====
<syntaxhighlight lang="bash">
SENSOR_ID=0 # 0 for CAM0 and 1 for CAM1 ports
FRAMERATE=60 # Framerate can go from 2 to 60 for 1920x1080 mode
gst-launch-1.0 nvarguscamerasrc sensor-id=$SENSOR_ID ! "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=$FRAMERATE/1" ! nvvidconv ! nvoverlaysink
</syntaxhighlight>

====4032x3040====

<syntaxhighlight lang="bash">
SENSOR_ID=0 # 0 for CAM0 and 1 for CAM1 ports
FRAMERATE=30 # Framerate can go from 2 to 30 for 4032x3040 mode
gst-launch-1.0 nvarguscamerasrc sensor-id=$SENSOR_ID ! "video/x-raw(memory:NVMM),width=4032,height=3040,framerate=$FRAMERATE/1" ! nvvidconv ! "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=$FRAMERATE/1" ! nvoverlaysink
</syntaxhighlight>

===MP4 Recording===

====1920x1080====
<syntaxhighlight lang="bash">
SENSOR_ID=0 # 0 for CAM0 and 1 for CAM1 ports
FRAMERATE=60 # Framerate can go from 2 to 60 for 1920x1080 mode
gst-launch-1.0 -e nvarguscamerasrc sensor-id=$SENSOR_ID ! "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=$FRAMERATE/1" ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=rpi_v3_imx477_cam$SENSOR_ID.mp4
</syntaxhighlight>

====4032x3040====

<syntaxhighlight lang="bash">
SENSOR_ID=0 # 0 for CAM0 and 1 for CAM1 ports
FRAMERATE=30 # Framerate can go from 2 to 30 for 4032x3040 mode
gst-launch-1.0 -e nvarguscamerasrc sensor-id=$SENSOR_ID ! "video/x-raw(memory:NVMM),width=4032,height=3040,framerate=$FRAMERATE/1" ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=rpi_v3_imx477_cam$SENSOR_ID.mp4
</syntaxhighlight>

===JPEG snapshots===

====1920x1080====
<syntaxhighlight lang="bash">
SENSOR_ID=0 # 0 for CAM0 and 1 for CAM1 ports
FRAMERATE=60 # Framerate can go from 2 to 60 for 1920x1080 mode
NUMBER_OF_SNAPSHOTS=20
gst-launch-1.0 -e nvarguscamerasrc num-buffers=$NUMBER_OF_SNAPSHOTS sensor-id=$SENSOR_ID ! "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=$FRAMERATE/1" ! nvjpegenc ! multifilesink location=%03d_rpi_v3_imx477_cam$SENSOR_ID.jpeg
</syntaxhighlight>

====4032x3040====

<syntaxhighlight lang="bash">
SENSOR_ID=0 # 0 for CAM0 and 1 for CAM1 ports
FRAMERATE=30 # Framerate can go from 2 to 30 for 4032x3040 mode
NUMBER_OF_SNAPSHOTS=20
gst-launch-1.0 -e nvarguscamerasrc num-buffers=$NUMBER_OF_SNAPSHOTS sensor-id=$SENSOR_ID ! "video/x-raw(memory:NVMM),width=4032,height=3040,framerate=$FRAMERATE/1" ! nvjpegenc ! multifilesink location=%03d_rpi_v3_imx477_cam$SENSOR_ID.jpeg
</syntaxhighlight>

{{ContactUs}}

[[Category:GStreamer]][[Category:Jetson]][[Category:JetsonNano]][[Category:JetsonTX2]][[Category:NVIDIA Xavier]][[Category:JetsonXavierNX]][[Category:Sony]][[Category:Jetson V4L2 Drivers]][[Category:NVIDIA Jetson Orin]][[Category:NVIDIA Jetson Orin Nano]]