Disk Encryption

🚧 Documentation under development

The RidgeRun Platform Security Manual guide is currently under active development. Some sections may be incomplete or change without notice.

Questions? Contact RidgeRun or email to support@ridgerun.com.

RidgeRun Platform Security Manual from RidgeRun

Contents Pin

Disk Encryption in Yocto with meta-tegra Layer

This wiki explain the Disk encryption can not be do completely implemented with yocto recipes, is necessary to include some after build process to insert data on the encrypt partitions. This wiki cover the changes necessaries for disk encryption in yocto to implement rootfs A/B encryption, user data partition encryption.

We will separate the wiki on two section, one for rootfs, and one for user data partitions.

Useful references

This repository include a example of how to do it using the Trusted App nvluks-srv. meta-rootfs-enc.
Arch linux wiki about crysetup. Encrypting devices with cryptsetup
Some considerations for OTA update with disk encryption can be found on: Mender OTA with disk encryption.

Disk encryption for rootfs

To implement disk encryption you will need to do 4 things:

Modify the initramfs recipe to include the decryption process
Modify the flash.xml to include the separated boot partition and encrypt tag
Update the eks.img with your encryption key
Encrypt the partition and write them to the jetson

Modify the initramfs recipe to include the decryption process

In the init ram filesystem you need to add the libs necessary for decrypt using nvluks-srv and crytpsetup.

To do that create a recipe append tegra-minimal-initramfs.bbappend.

PACKAGE_INSTALL:append = " libcrypto efivar cryptsetup libgcc optee-nvsamples openssl-conf openssl-ossl-module-legacy"

Note: is recommended keep the tegra-minimal-initramfs with minimal size.

Now you can modify the init-boot script to decrypt the partition. To do this you can create a new script or patch the existing. The final script looks like:

#!/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
mount -t proc proc -o nosuid,nodev,noexec /proc
mount -t devtmpfs none -o nosuid /dev
mount -t sysfs sysfs -o nosuid,nodev,noexec /sys
mount -t efivarfs efivarfs -o nosuid,nodev,noexec /sys/firmware/efi/efivars

rootdev=""
opt="rw"
wait=""
fstype="auto"

[ ! -f /etc/platform-preboot ] || . /etc/platform-preboot

if [ -z "$rootdev" ]; then
    for bootarg in `cat /proc/cmdline`; do
	case "$bootarg" in
	    root=*) rootdev="${bootarg##root=}" ;;
	    ro) opt="ro" ;;
	    rootwait) wait="yes" ;;
        rootfstype=*) fstype="${bootarg##rootfstype=}" ;;
	esac
    done
fi

if [ -n "$wait" -a ! -b "${rootdev}" ]; then
    echo "Waiting for ${rootdev}..."
    count=0
    while [ $count -lt 25 ]; do
	test -b "${rootdev}" && break
	sleep 0.1
	count=`expr $count + 1`
    done
fi
echo "Mounting ${rootdev}..."
[ -d /mnt ] || mkdir -p /mnt
count=0
while [ $count -lt 5 ]; do
    __l4t_enc_root_dm="l4t_enc_rootA1";
    __l4t_enc_root_dm_devA="/dev/mapper/${__l4t_enc_root_dm}"
    eval nvluks-srv-app -g -c "Partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p2 ${__l4t_enc_root_dm} 
    __l4t_enc_root_dm="l4t_enc_rootB2";
    __l4t_enc_root_dm_devB="/dev/mapper/${__l4t_enc_root_dm}"
    eval nvluks-srv-app -g -c "Partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p4 ${__l4t_enc_root_dm}
    if [ "$rootdev" != "/dev/mmcblk0p1" ]; then
    __l4t_enc_root_dm_dev=$__l4t_enc_root_dm_devB
    else
    __l4t_enc_root_dm_dev=$__l4t_enc_root_dm_devA
    fi
    if mount "${__l4t_enc_root_dm_dev}" /mnt; then
	break
    fi
    sleep 1.0
done

# Disable luks-srv TA
nvluks-srv-app -n > /dev/null 2>&1;

[ $count -lt 5 ] || exec sh

[ ! -f /etc/platform-pre-switchroot ] || . /etc/platform-pre-switchroot

echo "Switching to rootfs on ${rootdev}..."
mount --move /sys  /mnt/sys
mount --move /proc /mnt/proc
mount --move /dev  /mnt/dev
exec switch_root /mnt /sbin/init

In this new script the modification is in this section:

count=0
while [ $count -lt 5 ]; do
    __l4t_enc_root_dm="l4t_enc_rootA";
    __l4t_enc_root_dm_devA="/dev/mapper/${__l4t_enc_root_dm}"
    eval nvluks-srv-app -g -c "Partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p2 ${__l4t_enc_root_dm} 
    __l4t_enc_root_dm="l4t_enc_rootB";
    __l4t_enc_root_dm_devB="/dev/mapper/${__l4t_enc_root_dm}"
    eval nvluks-srv-app -g -c "Partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p4 ${__l4t_enc_root_dm}
    if [ "$rootdev" != "/dev/mmcblk0p1" ]; then
    __l4t_enc_root_dm_dev=$__l4t_enc_root_dm_devB
    else
    __l4t_enc_root_dm_dev=$__l4t_enc_root_dm_devA
    fi
    if mount "${__l4t_enc_root_dm_dev}" /mnt; then
	break
    fi
    sleep 1.0
done

# Disable luks-srv TA
nvluks-srv-app -n > /dev/null 2>&1;

In this section first we get the encryption passphrase from the Trusted App nvluks-srv-app and use it to unencrypt both rootfs partition using cryptseup.

    __l4t_enc_root_dm="l4t_enc_rootA";
    __l4t_enc_root_dm_devA="/dev/mapper/${__l4t_enc_root_dm}"
    eval nvluks-srv-app -g -c "Partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p2 ${__l4t_enc_root_dm} 
    __l4t_enc_root_dm="l4t_enc_rootB";
    __l4t_enc_root_dm_devB="/dev/mapper/${__l4t_enc_root_dm}"
    eval nvluks-srv-app -g -c "Partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p4 ${__l4t_enc_root_dm}

Next the decision of which boot is made based on the $rootdev values when not encryption is enable.

    if [ "$rootdev" != "/dev/mmcblk0p1" ]; then
    __l4t_enc_root_dm_dev=$__l4t_enc_root_dm_devB
    else
    __l4t_enc_root_dm_dev=$__l4t_enc_root_dm_devA
    fi
    if mount "${__l4t_enc_root_dm_dev}" /mnt; then
	break
    fi

Finally nvluks-srv-app is disabled.

    # Disable luks-srv TA
    nvluks-srv-app -n > /dev/null 2>&1;

Disable this trusted app is necessary to ensure the chain of trust is protected. Also keep in mind that if secure boot is not enable any can modify the boot process to retrieve the encryption passphrase.

Modify the flash.xml to include the separated boot partition and encrypt tag

You will need to modify the flash.xml the expected xml should define the rootfs partitions like:


        <partition name="primary_gpt" type="primary_gpt">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 19968 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 8 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
            <description> **Required.** Contains primary GPT of the `sdmmc_user` device. All
              partitions defined after this entry are configured in the kernel, and are
              accessible by standard partition tools such as gdisk and parted. </description>
        </partition>
        <partition name="APP" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 46383104 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <align_boundary> 4096 </align_boundary>
            <percent_reserved> 0 </percent_reserved>
            <filename> boot-folder.ext4 </filename>
            <unique_guid>  </unique_guid>
            <description> **Required.** Contains the rootfs boot file. This partition must be defined
              after `primary_GPT` so that it can be accessed as the fixed known special device
              `/dev/mmcblk0p1`. </description>
        </partition>
        <partition name="APP_enc" type="data" encrypted="true">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 7241220096 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <align_boundary> 4096 </align_boundary>
            <percent_reserved> 0 </percent_reserved>
            <filename> APPFILE </filename>
            <unique_guid>  </unique_guid>
            <description> **Required.** Contains the rootfs. This partition must be defined
              after `primary_GPT` so that it can be accessed as the fixed known special device
              `/dev/mmcblk0p2`. </description>
        </partition>
        <partition name="APP_b" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 46383104 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <align_boundary> 4096 </align_boundary>
            <percent_reserved> 0 </percent_reserved>
            <filename> boot-folder.ext4 </filename>
            <unique_guid>  </unique_guid>
            <description> **Required.** Contains the rootfs. This partition must be defined
              after `primary_GPT` so that it can be accessed as the fixed known special device
              `/dev/mmcblk0p3`. </description>
        </partition>
        <partition name="APP_b_enc" type="data" encrypted="true">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 7241220096 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <align_boundary> 4096 </align_boundary>
            <percent_reserved> 0 </percent_reserved>
            <filename> APPFILE_b </filename>
            <unique_guid>  </unique_guid>
            <description> **Required.** Contains the rootfs. This partition must be defined
              after `primary_GPT` so that it can be accessed as the fixed known special device
              `/dev/mmcblk0p4`. </description>
        </partition>

To do that you can create a tegra-binaries.bbappend with

# Custom flash partition setup for L4T 
FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
SRC_URI += "file://flash_l4t_enc.xml"

do_preconfigure:append() {
    cp ${WORKDIR}/flash_l4t_enc.xml ${S}/
}

and define your flash layout in your machine with:

# Use flash layout template
PARTITION_LAYOUT_TEMPLATE = "flash_l4t_enc.xml"

The change made on the flash layout are:

         <partition name="APP" type="data">
             <allocation_policy> sequential </allocation_policy>
             <filesystem_type> basic </filesystem_type>
-            <size> 7287603200 </size>
+            <size> 46383104 </size>
+            <file_system_attribute> 0 </file_system_attribute>
+            <allocation_attribute> 0x8 </allocation_attribute>
+            <align_boundary> 4096 </align_boundary>
+            <percent_reserved> 0 </percent_reserved>
+            <filename> boot-folder.ext4 </filename>
+            <unique_guid>  </unique_guid>
+            <description> **Required.** Contains the rootfs boot file. This partition must be defined
+              after `primary_GPT` so that it can be accessed as the fixed known special device
+              `/dev/mmcblk0p1`. </description>
+        </partition>
+        <partition name="APP_enc" type="data" encrypted="true">
+            <allocation_policy> sequential </allocation_policy>
+            <filesystem_type> basic </filesystem_type>
+            <size> 7241220096 </size>
             <file_system_attribute> 0 </file_system_attribute>
             <allocation_attribute> 0x8 </allocation_attribute>
             <align_boundary> 4096 </align_boundary>

Where the APP partition was divided in two, one unencrypted used to boot and other new partition encrypted for the rootfs.

Update the eks.img with your encryption key

For disk encryption you will need to create a encryption key and package it with your fuse key inside a key blob. To do that you can use the gen_ekb.py tool.

For the jetson xavier nx a use example is:

echo "${FV_32}" > fv_ekb_t194   # (Fixed Vector) 32 hex chars (16 bytes)
openssl rand -hex 16 > disk_enc.key    # (SMK_2) produces 32 hex chars (16 bytes)
python3 "tools/gen_ekb.py" -chip t194 \
  -kek2_key kek2.key \
  -fv fv_ekb_t194 \
  -in_sym_key2 disk_enc.key \
  -out eks.img

This script generate a eks.img that should replace the eks.img on your tegraflash instance. The gen_ekb tool is provided by nvidia.

For this use case the fixed vector used should be the same that is used on the op-tee compilation. To generate the kek2.key you can review the secure boot documentation. https://developer.ridgerun.com/wiki/index.php/RidgeRun_Platform_Security_Manual/Getting_Started/Secure_Boot/NVIDA-Jetson

It is possible to use the default values for the kek2.key in case secureboot is not desired.

Encrypt the partition and write them to the jetson

For flashing you will need two images, the boot folder image and the encrypted APPFILE image. You can create the boot folder image from a yocto recipe or while encrypting the APPFILE image.

The encryption process is made as an after build script because the loop and dm-crypt driver of the host are used and yocto lack the permissions needed.

An example script to make it looks like:

#Create the image files with a defined size
truncate -s 7051001856 ${__rootfs_image_name}
truncate -s 46383104 ${__boot_folder_img_name}

#Define the encrypt command using nvidia tool
GEN_LUKS_PASS_CMD="tools/gen_luks_passphrase.py  -g -c '${__rootfsuuid}' -k ${SMK_2} "
#Encrypt the rootfs image
eval ${GEN_LUKS_PASS_CMD} | sudo cryptsetup \
		--type luks2 \
		-c aes-xts-plain64 \
		-s 256 \
		--uuid "${__rootfsuuid}" \
		luksFormat \
        ${__rootfs_image_name}

#open the encrypted partition to give it the correct format
eval ${GEN_LUKS_PASS_CMD} | sudo cryptsetup luksOpen ${__rootfs_image_name} ${__l4t_enc}

#mount original and encrypted rootfs 
sudo mount /dev/mapper/${__l4t_enc} ${__enc_rootfs_mountpoint}
sudo mount  ${__original_rootfs} ${__rootfs_original_mountpoint}

#fill boot folder image
mkfs.ext4 -F -i 4096 -b 4096  ${__boot_folder_img_name}  -d ${__rootfs_original_mountpoint}/boot/

#fill enc_rootfs
sudo mkfs.ext4 /dev/mapper/${__l4t_enc} -d ${__rootfs_original_mountpoint}/
sleep 5 # give time to host to make a fsck before unmount
sudo umount ${__enc_rootfs_mountpoint}
sudo cryptsetup luksClose /dev/mapper/${__l4t_enc}

In this case the boot image is generated on the script. To use this encrypted images you just need to replace the original on your tegraflash. The gen_luks_passphrase.py tool is provided by nvidia.

Disk Encryption for user data partitions

To implement disk encryption for user data partitions you will need to do the same things that were :

Modify the initramfs recipe to include the decryption process
Modify the flash.xml to include the encrypt tag and the encrypted image
Update the eks.img with your encryption key
Encrypt the partition and write them to the jetson

The third step is the same that the applied for rootfs and only need to be made one time for tegraflash instance.

Modify the initramfs recipe to include the decryption process

You need to extend the init-boot script to decrypt the user data partition. You can add the partition decrypt with:

+    __l4t_enc_dm="user_data";
+    __l4t_enc_dm_user="/dev/mapper/${__l4t_enc_root_dm}"
+    eval nvluks-srv-app -g -c "user_partition_uuid" | cryptsetup luksOpen /dev/mmcblk0p5 ${__l4t_enc_root_dm}

If you need to mount the partition on boot you can also add it in the init-boot script with:

mount "${__l4t_enc_dm_user}" /mnt/data

Keep in mind that for the mount to work successfully it need to have the folder direction where to mount, and that the rootfs is mounted on /mnt. The easy way to mount is to create a empty folder in the rootfs where the user data will be mounted and mount the data partition after rootfs is mounted to /mnt.

Modify the flash.xml to include the separated boot partition and encrypt tag

For user data partitions you can include the new partition on the flash layout as any other partition, just include the filename of the encrypted image.

        <partition name="Data_enc" type="data" encrypted="true">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 104857600 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <align_boundary> 4096 </align_boundary>
            <percent_reserved> 0 </percent_reserved>
            <filename> user-folder.ext4 </filename>
            <unique_guid>  </unique_guid>
            <description> User data partition </description>
        </partition>

And include the image on the TEGRA_BUPGEN_STRIP_IMG_NAMES in your machine configuration to avoid the signing before it is make.

# Avoid sign user image on initram

TEGRA_BUPGEN_STRIP_IMG_NAMES = "user-folder.ext4"

Encrypt the partition and write them to the jetson

To create a image from a yocto recipe you can create a bbappend to your image recipe like this.

separate_user_folder() {
    [ -d "${WORKDIR}/user" ] && rm -rf "${WORKDIR}/user"
    mv ${WORKDIR}/rootfs/user ${WORKDIR}
    mkdir ${WORKDIR}/rootfs/data #make point to mount
}

create_user_image() {
    USER_IMAGE_SIZE="104857600" #100MB
    USER_IMAGE_PATH="${IMGDEPLOYDIR}/user-image-${MACHINE}${IMAGE_VERSION_SUFFIX}${IMAGE_NAME_SUFFIX}.${IMAGE_TEGRAFLASH_FS_TYPE}"
    USER_LINK_PATH="${IMGDEPLOYDIR}/user-image-${MACHINE}${IMAGE_NAME_SUFFIX}.${IMAGE_TEGRAFLASH_FS_TYPE}"

    truncate -s "${USER_IMAGE_SIZE}" "${USER_IMAGE_PATH}"
    mkfs.ext4 -F -i 4096 -b 4096 $USER_IMAGE_PATH  -d ${WORKDIR}/user
    # Error codes 0-3 indicate successfull operation of fsck (no errors or errors corrected)
    fsck.ext4 -pvfD $USER_IMAGE_PATH  || [ $? -le 3 ]

    ln -rsf $USER_IMAGE_PATH $USER_LINK_PATH
}


tegraflash_custom_pre() {
    local USER_LINK_PATH="${IMGDEPLOYDIR}/user-image-${MACHINE}${IMAGE_NAME_SUFFIX}.${IMAGE_TEGRAFLASH_FS_TYPE}"
    cp $USER_LINK_PATH ./user-image.${IMAGE_TEGRAFLASH_FS_TYPE}
}

IMAGE_PREPROCESS_COMMAND += "separate_user_folder; "
IMAGE_PREPROCESS_COMMAND += "create_user_image; "

This append will create a user-image.ext4 image on the tegraflash instance that can be processed to create a encrypted version, for that you can add to you encrypting script:

GEN_LUKS_PASS_CMD="$(build_gen_luks_cmd "${<user_partition_uuid>}")"

eval ${GEN_LUKS_PASS_CMD} | sudo cryptsetup \
		--type luks2 \
		-c aes-xts-plain64 \
		-s 256 \
		--uuid "${user_partition_uuid}" \
		luksFormat \
        ${__data_img_name}
eval ${GEN_LUKS_PASS_CMD} | sudo cryptsetup luksOpen ${__data_img_name} ${__l4t_usr_data_enc}

sudo mount  ${__original_data} ${__original_user_data_mountpoint}
sudo mkfs.ext4 /dev/mapper/${__l4t_usr_data_enc} -d ${__original_user_data_mountpoint}/
sleep 5
sudo cryptsetup luksClose ${__l4t_usr_data_enc}
sudo umount  ${__original_data}

After that you can proceed with build and flashing.

❯