How to Create Yocto Projects - Yocto Utils
Yocto Utils
This section will guide you through a set of tools that will make your development in Yocto easier, faster and more comfortable.
Repo
Repo is a tool built on top of Git. Repo helps manage many Git repositories, does the uploads to revision control systems, and automates parts of the development workflow. Repo is not meant to replace Git, only to make it easier to work with Git. The repo command is an executable Python script that you can put anywhere in your path.
Install repo
You can install it using apt:
sudo apt-get install repo
Or downloading it directly
1. Add repo script to bin directory in home.
mkdir -p ~/bin curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo chmod a+x ~/bin/repo
2. Append the directory if it's not already in PATH
PATH=$PATH:~/bin
How to use repo with Yocto
Repo uses an XML manifest in order to enumerate multiple remotes, repositories, branches and revisions, then it fetches them. For example, using a Boundary Devices repository with a manifest we can use:
cd <ANY_DIR> repo init -u https://github.com/boundarydevices/boundary-bsp-platform -b dunfell
With these options being the most important:
-u URL, --manifest-url=URL : manifest repository location -b REVISION, --manifest-branch=REVISION : manifest branch or revision (use HEAD for default) -m NAME.xml, --manifest-name=NAME.xml: initial manifest file (defaults to default.xml), you can also use this to specify a completely different manifest in your system and it will be used.
After doing repo init, you can sync it using:
repo sync
This will generate a directory structure with the given repositories defined in the XML manifest, normally in Yocto it will create a sources directory with each layer, and a setup script to create a build.
Analyzing the repo manifest
Take as an example the manifest located in: https://github.com/boundarydevices/boundary-bsp-platform/blob/gatesgarth/default.xml
Defaults
The default settings are set as:
<default sync-j="4" revision="gatesgarth"/>
Remotes
Remotes are defined as below, they are named to be used later for each project:
<remote fetch="https://git.yoctoproject.org/git" name="yocto"/> <remote fetch="https://github.com/Freescale" name="freescale"/> <remote fetch="https://github.com/openembedded" name="oe"/> <remote fetch="https://github.com/boundarydevices" name="boundary"/>
Projects
Project are basically the repositories to fetch, they will use the remote defined above, the name on the hosting server, and then clone them into a path. You can also use the revision which is the commit to checkout, and upstream which is the branch to use, this is not necessarily the same for all projects obviously.
<project remote="yocto" revision="6bd890d9e011014cf323e61267f8b256949d44aa" upstream="pyro" name="poky" path="sources/poky"/> <project remote="yocto" revision="83b779a7ef69006564fb8d27e65c31917bc3b6f0" upstream="pyro" name="meta-freescale" path="sources/meta-freescale"/> <project remote="oe" revision="5e82995148a2844c6f483ae5ddd1438d87ea9fb7" upstream="pyro" name="meta-openembedded" path="sources/meta-openembedded"/> <project remote="freescale" revision="a73a14247afbbcf840bab7b76f953004fe109253" upstream="pyro" name="meta-freescale-3rdparty" path="sources/meta-freescale-3rdparty"/> <project remote="freescale" revision="cd5c7a2539f40004f74126e9fdf08254fd9a6390" upstream="pyro" name="meta-freescale-distro" path="sources/meta-freescale-distro"/>
You can also perform operations like copy files from the project directly after syncing:
<project remote="boundary" name="boundary-bsp-base" path="sources/base"> <copyfile dest="README" src="README"/> <copyfile dest="setup-environment" src="setup-environment"/> </project>
Creating a manifest
You can create a manifest of your current build by using:
repo manifest -r -o my_manifest.xml
Devtool
Let's assume we are building for qemuarm64 and you have set the `$DEVDIR` variable.
First use devtool to set a new workspace for the recipe:
devtool modify gst-perf
Source code now will be located in the new created folder:
ls $BUILDDIR/workspace/sources/gst-perf
When you compile and install this recipe, these sources will be used, you can check it by running:
bitbake -c install gst-perf # Compilation and installation tree $BUILDDIR/tmp/work/aarch64-poky-linux/gst-perf/1.0-r0/image # Files installed
Devtool deploy
deploy method is agnostic to the platform and any paths, it takes into account everything and just needs the name of the recipe to update it in the target by using SSH
Setting up
cd $BUILDDIR runqemu qemuarm64 core-image-training nographic
Let's get the IP from the QEMU instance, inside the instance:
ifconfig eth0 | grep "inet " | awk -F'[: ]+' '{ print $4 }'
Save this IP, we will use it later.
Testing
Modify the source code, for example the Description of the plugin, and then build again:
bitbake -c install gst-perf
Then you can deploy the changes using devtool:
devtool deploy-target gst-perf root@$QEMU_IP
You can see the new version by running in the QEMU:
ldconfig
gst-inspect-1.0 perf | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" | grep Description -m 1
You can also undeploy the changes
undeploy-target gst-perf root@$QEMU_IP
Saving changes
To save the changes
devtool update-recipe --append=$DEVDIR/sources/meta-training/ gst-perf devtool reset gst-perf
Equivalent to:
devtool finish gst-pref $DEVDIR/sources/meta-training/
Both commands will just save the changes made in the new commits made for the branch inside the $BUILDDIR/workspace/sources/ folder
NFS
NFS method let's you mount the filesystem in your host and QEMU will load it, allowing changes to it in real time. Reference: https://docs.yoctoproject.org/dev-manual/qemu.html#running-under-a-network-file-system-nfs-server
Setting up
# Set variables cd $BUILDDIR export NFS_ROOT=`pwd`/test-nfs # Install dependencies for NFS bitbake and QEMU support sudo apt-get install rpcbind bitbake meta-ide-support # Set up NFS and start QEMU instance runqemu-extract-sdk ./tmp/deploy/images/qemuarm64/core-image-training-qemuarm64.tar.bz2 $NFS_ROOT runqemu qemuarm64 $NFS_ROOT nographic
Now, every change to the filesystem inside `$NFS_ROOT` will be seen in real time on the QEMU instance running
Testing
Modify the source code, and then:
cd $BUILDDIR bitbake -c install gst-perf cp -r ./tmp/work/aarch64-poky-linux/gst-perf/1.0-r0/image/* $NFS_ROOT/
Stopping
To stop the NFS sharing use:
runqemu-export-rootfs stop $NFS_ROOT
Then remove the folders
rm -rf $NFS_ROOT rm -rf $NFS_ROOT.pseudo_state
Devtool for Kernel Development
Modify
Setup the kernel sources:
devtool modify virtual/linux
Modify $BUILDDIR/workspace/sources/linux-yocto/init/calibrate.c by adding these printk to the calibrate_delay function.
void calibrate_delay(void)
{
unsigned long lpj;
static bool printed;
int this_cpu = smp_processor_id();
printk("*************************************\n");
printk("* *\n");
printk("* HELLO YOCTO KERNEL *\n");
printk("* *\n");
printk("*************************************\n");
if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
Deploy
We need to copy the image /boot/
scp tmp/work/qemuarm64-poky-linux/linux-yocto/5.4.153+git999-r0/image/boot/Image root@<QEMU_IP>:/boot/
Then exit Qemu, and enter again. You should see the message from above while running:
dmesg | less
Devshell
Setting up
Remember the $DEVDIR folder? Get into there and get into the build directory if you haven't done so.
cd $DEVDIR source ./sources/poky/oe-init-build-env training-build/
We are going to modify the perf element. So get into the board and check the current Description for this element.
gst-inspect-1.0 perf | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" | grep Description -m 1
Usage
If you added gst-perf to the workspace using devtool, first remove it: devtool reset gst-perf.
Let's start using devshell, execute:
bitbake gst-perf -c devshell
This will open up the sources on the $S variable, normally $WORKDIR/git [1] if using git.
Updating the code
0. Check if the source code is using git or not, if not and you are planning to make changes that you are planning to capture you will need a new git local repo or to use with quilt, in any case you need to either start the repo or create a new quilt patch before modifying the files. More information in the Saving changes section below.
1. Modify the code, for example update the Description for the element:
Use vim or your editor, and search for gst_element_class_set_static_metadata function. Modify it.
Compiling
1. Run by hand:
./autogen.sh ./configure --prefix `pwd`/../image/usr/ --libdir `pwd`/../image/usr/lib --build=x86_64-linux --host=aarch64-poky-linux make && make install
Then all the will be installed at first level inside
tree ../image/
../image/
└── usr
└── lib
└── gstreamer-1.0
├── libgstperf.la
└── libgstperf.so
Testing
You can test by updating the binaries and files in the board or in the QEMU instance or
scp -r ../image/* root@192.168.7.2:/
Look that the copies need to be done by hand, different to devtool.
Then run on the board:
ldconfig
gst-inspect-1.0 perf | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" | grep Description -m 1
Saving changes
Create the patch
git
Create patch, if using git for example:
git add plugins/gstperf.c git commit -m "update" git format-patch HEAD~1
quilt
quilt new 0001-update.patch quilt add plugins/gstperf.c edit plugins/gstperf.c
Modify file and then
quilt refresh
Now you can copy the quilt generated file.
Update recipe with patch
Save the path of the patch file, using pwd and the name of it:
echo `pwd`/0001-update.patch
Update recipe with patch:
cd $DEVDIR/sources/meta-training/recipes-multimedia/gstreamer mkdir files cp $PATCH_PATH files/
And append it to the SRC_URI for the recipe manually.
cd $DEVDIR/sources/meta-training/recipes-multimedia/gstreamer/
And then add to the gst-perf_%.bbapped:
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI += "file://0001-update.patch \
"
Now to test you may copy the values from the image folder in the workdir, first get this value doing a grep:
bitbake -e gst-perf | grep ^D=
Then just the same copy and test commands.
NOTE: Take a look at how many manual steps are needed in order to achieve the same that the devtool is doing for us.
run.do_* scripts
Setting up
Remember the $DEVDIR folder again? Get into there and get into the build directory if you haven't done so.
cd $DEVDIR source ./sources/poky/oe-init-build-env training-build/
We are going to modify the perf element again. So get into the board and check the current Description for this element.
gst-inspect-1.0 perf | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" | grep Description -m 1
Usage
To start, we need to get into the code of the package, for this check where it is located by using:
bitbake -e gst-perf | grep ^S=
And move to the directory it is pointing.
Updating the code
0. Check if the source code is using git or not, if not and you are planning to make changes that you are planning to capture you will need a new git local repo or to use with quilt, in any case you need to either start the repo or create a new quilt patch before modifying the files. More information in the Saving changes section of devshell, same applies here.
1. Modify the code, for example update the Description for the element:
Use vim or your editor, and search for gst_element_class_set_static_metadata function. Modify it.
Compiling
This is the easy part, you can just run the compilation and install scripts autogenerated by bitbake:
../temp/run.do_compile ../temp/run.do_install
If these scripts don't exist, execute the install task for the desired recipe.
Testing
You can test by updating the binaries and files in the board or in the QEMU instance or
scp -r ../image/* root@192.168.7.2:/
Look that the copies need to be done by hand, different to devtool.
Then run on the board:
ldconfig
gst-inspect-1.0 perf | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" | grep Description -m 1
Yocto Extensible SDK
Building Yocto can take time. There's an alternative for bigger teams working on a Yocto image, so that only one person builds Yocto, and the rest of the team uses the resulting build to develop their applications. This is the so-called Yocto SDK.
This way, Yocto SDK provides a simpler environment to application engineers, to avoid all the complexity and time spent to build the whole Yocto image.
Yocto provides two SDK types: Standard and Extensible. We will build the extensible SDK, as it contains more useful features as can be seen in the following table:
| Feature | Standard SDK | Extensible SDK |
|---|---|---|
| Toolchain | Yes | Yes |
| Debugger | Yes | Yes |
| Size | 100+MB | 1+GB |
| devtool | No | Yes |
| Build Images | No | Yes |
| Updateable | No | Yes |
| Managed Sysroot | No | Yes |
| Installed Packages | No | Yes |
| Construction | Packages | Shared State |
Build the SDK
To build the Extensible SDK:
bitbake core-image-training -c populate_sdk_ext
Install the SDK
The SDK can be found inside the $BUILDDIR/tmp/deploy/sdk directory.
To install the SDK:
cd $BUILDDIR ./tmp/deploy/sdk/poky-glibc-x86_64-core-image-training-aarch64-qemuarm64-toolchain-ext-3.1.11.sh
You will be asked for the directory where the SDK will be installed. From now on, we are going to call this directory $SDK_DIR
Initialize the SDK
Every time you want to use the SDK, you have to source the environment script:
cd $SDK_DIR source environment-setup-aarch64-poky-linux
Check that you get the following text after the source command:
SDK environment now set up; additionally you may now run devtool to perform development tasks. Run devtool --help for further details.
Extended SDK with Devtool
First we need to create the image with
devtool build-image core-image-training
To add the gst-perf sources to the workspace, run:
devtool modify gst-perf
Now you can modify it at:
workspace/sources/gst-perf/
Then, to build the changes:
devtool deploy-target gst-perf root@$QEMU_IP
devtool knows which files need to be updated and where they need to be installed, so you don't have to worry.