RidgeRun 2011Q2 SDK Development and Integration Guide

From RidgeRun Developer Connection

(Difference between revisions)
Jump to:navigation, search
m (Application installation process overview)
Line 260: Line 260:
** '''fs/''': The target's file system. Through the makefile syntax, which will be expanded upon later, whatever should be ultimately stored in the target's filesystem from the output stored in the fs-dev directory is specified and sent to the appropriate directory here once installation has been completed (2).
** '''fs/''': The target's file system. Through the makefile syntax, which will be expanded upon later, whatever should be ultimately stored in the target's filesystem from the output stored in the fs-dev directory is specified and sent to the appropriate directory here once installation has been completed (2).
* '''bsp/'''
** '''local/''': This directory contains the raw output of the installation processes for applications specified through the fs/host-apps directory. As mentioned before, these are packages that for some reason require to be present in the host system.
** '''local/''': This directory contains the raw output of the installation processes for applications specified through the fs/host-apps directory. As mentioned before, these are packages that for some reason require to be present in the host system.

Revision as of 15:46, 18 April 2011


Linux Application Development

This section targets software developers with a little background on Linux software development, presenting some basic concepts about Linux applications that may be unfamiliar to developers with experience in real time operating systems (RTOS).

Linux application development is more like standard PC application development and less like RTOS focused development. The main differences is the clear separation between the applications and the operating system and also that applications are stored on a file system.

This section doesn't replace a good book on Linux/Unix software development, but introduces the minimal concepts that will be useful when porting applications.

Kernel and User Space

Linux is a multi-tasking operating system with protection between executing processes.Therefore applications run in their own address-space which is provided by the kernel using the hardware memory management unit (MMU).

Since applications are separate from the kernel, they live on some file system, therefore Linux needs a least one file system holding the initial applications: the root file system.

The File System

Linux file system is based on Unix file system concepts, and therefore it is composed of a tree of files that are located on some physical storage. The physical storage should provide the root of the tree (the root file system) and other devices can be “mounted” at directories in the file system tree.

Following the Unix mindset, most of the computer universe is represented like a file: files are data containers for more than binary executable files or raw data stored on memory devices, files are also used to represent device drivers, interprocess pipes, or system status.

Unix file system layout is also important, as there are standard locations for configuration files, shared libraries, binaries, shared data, etc.

Pseudo file systems

Linux has several pseudo file systems that when mounted don't reflect actual stored files on some memory device, but file handlers that can provide information about the system, a process, a driver, etc. This pseudo file systems generate the file entries and contents dynamically, based on the system status.

The two more common pseudo-file systems are procfs, and sysfs. More information on these can be found on the next section.

Linux File System Layout

Understanding the Linux file system layout is important for the porting process. The following table explains what's the purpose of the most common locations and should provide a guideline on where to store the contents of your application.

Intended for executable binaries required for the basic system.
Intended for executable binaries required for the basic system that should be available only to the super (root) user.
Shared libraries for the basic system.
Text configuration files of the basic system.
Temporary files: on most embedded systems this directory contents are lost between power cycles.
Intended to store data from system programs: web server pages, log files, mail server data, etc.
This is the typical mount point of Linux's pseudo file system procfs, which creates a set of files that contain different aspects of the running system, processes, drivers, etc.
This is the typical mount point of Linux's pseudo file system sysfs, which provides information on, and control of, devices drivers and kernel objects.
Intended for applications that don't form part of the standard system provided by the vendor. These are usually third-party applications. Most of the applications at this location don't conform to the bin/lib separation of the Unix file systems.
Intended for executable binaries that are not required for the basic system.
Intended for executable binaries that are not required for the basic system that should be available only to the super user.
Shared libraries not required for the basic system.
Embedded Linux File System

The standard Linux file system is designed for use by more than one person at a time. This is rarely the case on an embedded device, so the difference between /bin, /sbin, /usr/bin, /usr/sbin, /opt may seem less useful than for standard desktop computing purposes.

However one of the important uses of this separation is that it simplifies the final layout of the file systems used by the embedded target. RidgeRun applies the following rules to decide where to locate the applications:

This layout allows for different mount points to be used for the /usr and /opt directories which in turn allows:

Embedded target file system

RidgeRun SDK creates the target file system on the Linux host machine, and processes it to create the proper target file system image to be downloaded (if required) to the embedded system. The file system of the target system is found under the base (referred to as $DEVDIR) of your SDK: $(DEVDIR)/fs/fs.

This directory only exists after the SDK has been built for the first time, and files can be copied to this location to include them in the target file system image.

Since the file system of the target should have a minimal footprint, the SDK only puts the strictly necessary parts of the applications into this directory, leaving out some run-time unnecessary parts of many programs like include files, manual and info pages, unused localizations, etc. However some of these files (like the includes) may be required during the compilation process for other programs. For this propose the SDK provides the directory $DEVDIR/fs/fsdev, which is the typical installation point of most applications (including all the unnecessary run-time files), and later the application scripts copy the minimal parts from $(DEVDIR)/fs/fsdev to $(DEVDIR)/fs/fs . More information on this procedure will be explained later on this document.

SDK build system

Building software for embedded systems is a complex task as it typically involves cross-compiling. Cross compiling presents new challenges to already established procedures for building software. The RidgeRun SDK uses a build system that simplifies the complexities of cross compiling.

The SDK build system was designed by embedded Linux programmers for embedded Linux programmers, therefore it not only simplifies the process of building embedded Linux applications, but the process of integrating, using or developing open source tools or packages with the SDK.

Introduction to RidgeRun SDK build system

Embedded Linux software development differs in various ways from the two environments it mixes: “embedded” and “Linux”.

Embedded Linux differs from most embedded environments using an RTOS in that Linux uses a file system outside the kernel, and advanced software tools like dynamic loaded libraries, multiple applications, dynamic modules, powerful networking tools and high modularity are supported.

An often overlook but crucial difference between Linux and other RTOS systems has to do with the dynamics of building and integrating open source software. Open source software is build by different communities around the world, and a typical embedded Linux system will include pieces from hundreds of developers and several different open source projects. This factor impacts various processes: system integration, application building, bug reporting, etc.RidgeRun support services minimizes this complexity by providing a single point to get access to embedded Linux support experts, but also by accounting this factor on the design of the build system.

Embedded Linux also differs from standard Linux distributions in its requirements for custom tailored kernel, drivers, and file system. Embedded limitations on footprint, power consumption and processing power create a big gap between embedded and desktop Linux systems.

All the previously mentioned details need to be taken into account by the development procedures for building and packaging the software for embedded Linux. We found three different approaches from SDK providers to build the system (with RidgeRun using the last option):

Binary software distributions: some embedded Linux providers ship the file system applications pre-compiled for the target hardware. This approach provides the advantage that the overhead and work to recompile the basic system is avoided, however also presents several disadvantages:

Source software distributions by emulation: some embedded Linux providers use a source building based system that is compiled by running a virtual machine emulating the target hardware and using the compiler natively on the emulated target. This approach has valuable advantages:

The disadvantages of this system are:

Source software distributions by cross compilation: consists on building the applications from source for the target hardware from the host target. The advantages are:

The disadvantages of this system are:

RidgeRun SDK build system is a source distribution built using cross compilation, and has minimized the complexities of it by defining a centralized system to simplify tool usage, the setting of compilation flags, dependency handling and minimizing host tools requirements.

SDK build tools

The SDK build system is based on a set of technologies commonly found on standard Linux developer machines:

The set of tools were selected are generally available on most systems, and can be extended to integrate with more sophisticated IDEs like Eclipse, or can be used standalone.

The SDK requires a Linux host machine for development, RidgeRun recommends Ubuntu. Please consult the SDK User Guide or RidgeRun support for the latest information on supported distributions.

Anatomy of a build system

An embedded Linux build system can be analyzed by how well it supports the requirements and the components used to meet those requirements. The RidgeRun SDK build system is composed by:

Centralized configuration system: a single system where the user can control the features and components of the software to be build.

Centralized build system: a single system that is responsible for building the software according to the configuration files generated by the configuration tool.

Application build tools: a set of tools and guidelines that provided support for target applications build process (as described in this document), such:

SDK Build system basics

The SDK build system needs to be understood to easily integrate new applications and port existing applications.

Configuration system

The SDK uses a dynamic configuration utility, called kconfig, which is shared by other important projects of the open source community, for example the Linux Kernel and the busybox set of tools.

The kconfig system used by RidgeRun differs from standard versions:

The build system provides a way to add menus to the configuration system and associate that value to a variable that can later be accessed by the Makefiles to effect decisions or translate it as a compile definition. For example the definition:

   config FS_APPS_MYAPP
         bool “myapp”

will create a definition:


if selected as true on the configuration system. Note that the suffix “CONFIG_” was added to the variable name. Those definitions can be used by the Makefiles by including the file $DEVDIR/bsp/mach/Make.conf

For more information on the syntax of the configuration files used by the kconfig system and the different options please consult $DEVDIR/bsp/bspc/kconfig-language.txt.

Build stages

The SDK build system builds difference components and this order may be important when building software that relays on other definitions or libraries. The SDK build order is:

The applications from the file system are separated in different directories, and are build in the following order:

The order in which the directories are built inside the specific fs stages defaults to alphabetically sorted directory names. If some application requires a dependent code to be build first, the order can be changed by using the metainfo system explained later on this document.

Makefile integration with the configuration system

The different makefiles of the system can be integrated with the configuration system by including a couple of files providing:

All three are described in detail throughout the rest of this document.

Integrating applications in the SDK

Customers are encouraged to create their applications in the $DEVDIR/myapps directory. Use of the other file system build points is not recommended. For example if you want to create a newapp application, you should proceed to create a directory $DEVDIR/myapps/newapp/.

Your application needs some elements in order for the application to be compiled by the SDK:

In this section we will guide you through the process of creating these files for the newapp application. It is recommended that you read the SDK User Guide before reading this section, and also have used and familiarized yourself with the SDK.

Application installation process overview

In order to understand the application installation process that occurs before one is finally compiled and stored in the target file system, it's good to have a grasp on some of the directories found within the "fs" directory on the development directory (devdir) that's being worked on.

Application installation process overview.

Integration with the configuration system (Or: The "config" file)

To integrate your application with the configuration system you have to create a file named Config in the application directory. This file contains logic to select if your application should be built within the SDK. A simple example file is:

        bool “NewApp Customer Application”
                Build my NewApp program

In addition, you can use the Config file to force the selection of other configuration options on the SDK (kernel features, libraries, etc). To do this all you need to do is to get the string of the definition you want to enable and add a select statement for it (shown later).

For example, if your application requires zlib (an open source application for compression) then you may use the following code to force incluing of zlib if your application is selected:

        bool “NewApp Customer Application”
        select FS_APPS_ZLIB
                Build my NewApp program

The name of the zlib configuration variable may be queried using the “help” of the feature in the configuration menu system. For example the help of “File System Configuration -> Select target's file system software -> SDK Applications -> zlib 1.2.3” says CONFIG_FS_APPS_ZLIB (just strip the prefix CONFIG_).

For further information about the syntax of this file check $DEVDIR/bsp/bspc/kconfig-language.txt. You can look at other Config files in the SDK to find one that is similar to your needs and use it as your starting point.

Once the Config file is created, you can run the “make config” target on your system to enable the the new application to be built and installed in the target file system. A screen will appear as follow:


Creating your application Makefile

In order to build your application you need to create a standard Makefile file. The Makefile must be created in your application directory, and it requires the following rules to be integrated with the SDK build system:

For more information on makefiles syntax or usage, please check: http://www.gnu.org/software/make/manual/make.html

Makefile header definitions

Let's look at an standard header section of an application Makefile:

   # License Text
   # ...
   .PHONY: build chkconfig preconfig buildfs install clean sim
   ifeq ($(DEVDIR),)
   $(error ====== DEVDIR variable is empty, invoke this
   Makefile from the BSP root, or provide the path to it =====)
   include $(DEVDIR)/bsp/mach/Make.conf
   include $(DEVDIR)/fs/Apps.defs

The previous code block shows the primary required headers:

Make.conf definitions

As you notice from the previous step, all the makefiles of the system include the Make.conf file, which provides macro definitions that are useful when writing your makefiles and integrating them to build nicely with the SDK build system:


Use this variable in front of every rule of the makefile to make the rule silent when building the SDK. If make is invoked with VERBOSE=1 then the rule will be verbose on screen. Example:

newapp: newapp.c
       $(V) $(CC) -o newapp newapp.c 

Use this variable at the end of every rule of the makefile to redirect the standard input to the $(LOGFILE) if the verbose mode is not enabled. This produces the compiler messages to go into the log file and compile silently. Example:

newapp: newapp.c
       $(V) $(CC) -o newapp newapp.c $(QOUT) 

This is the absolute path of a file where the compilation information is redirected when verbose mode is not used. Typically it is /dev/null, but may be overwritten by the command line of the make invocation.

Use this rule to print messages on the SDK build process that are aligned on the build tree of the SDK (instead of the standard “echo”). Example:

newapp: newapp.c
       $(ECHO) “Building my NewApp”
       $(V) $(CC) -o newapp newapp.c $(QOUT)

This is used when you need to invoke makefile from other directory. If the verbose mode is enabled it will invoke the other makefile silently. Example:

       $(V)$(MAKE) -C somelibrary $(QOUT)

This is the location of the target file system, and can be used to install files on it. Example:

       $(V) install -m 755 newapp $(FSROOT)/bin 
  When compiling autotools based applications (see Porting autotools based  applications), this is the standard installation prefix for applications, and is useful for copying the binaries to the final file system $(FSROOT)

Makefile targets

The top makefile of any application needs to define a standard set of targets to respond to. In an application Makefile all the following targets must be defined:

The following is an example of the previous targets inside an application Makefile:

         <build commands for your app>
         <install commands for your app>
         <clean commands for your app>
     sim commands if your app supports simulation>

  # Empty but required targets
Compiler usage

In order to compile the files with the proper cross-compiler you can use the follow environment variables (imported from the Make.conf file):

For example if you want to compile your application hello.c for the target machine you may add the following rule to the Makefile:

   hello: hello.c
               $(V)$(CC) $(CFLAGS) -o hello hello.c $(QOUT)
Installing your application

To install your application the rule simply needs to copy the required files into the target file system using the install rule. The target file system may be accessed using the makefile variable FSROOT:

               $(V) install -m 755 hello $(FSROOT)/opt/hello

In this example we use the install program which copy the programs and set the file permissions at the same time.

Dependency handling in your makefile

Once you were able to write a simple application and are ready to start development of more complex programs, one of the most useful tools is makefile dependency handling (recompilation on demand of changed source files). This eliminates rebuilding object files unnecessarily.

However proper dependency handling is a complex issue and you can find several resources online that explain how to perform dependency handling with Makefiles. RidgeRun recommends using autotools for your applications (see Porting autotools based applications for more information on how to integrate autotools based applications with the SDK). Please consider using autotools – even though it require a higher learning curve. Autotools will help you avoid common mistakes and make your application better portability (and easier to support a simulator build).

Handling dependencies: using metainfo

Most open source application developers create their software in a modular design so it can be reused by other people or projects. This is the reason you can find an endless list of required components to build your applications or components that you need to port to the SDK in order to make the port you are most interested in to work.This job requires the SDK build system be capable of handling dependencies between the different components at compile time, providing the following services to the components:

The SDK build system provides the above functionality with an optional metainfo file in the application directory. Before the development of the metainfo system, many separated techniques were used to accomplish the same goals, they will be documented along the new procedure, but are considered deprecated.

Exploring the metainfo file

Let's suppose we are adding an application named newapp-1.0 to the SDK. We will assume that newapp-1.0 requires of the library libapp-1.0 to be build, and requires the open source program flex to be executed on the host during the compilation.

This applications will appear in its own directory, under $DEVDIR/myapps/, with the following structure:

Previous sections dealt with the Config and Makefile files; in this section the metainfo file will be explored.

Our first step will be to modify the Config file of newapp-1.0 to add a line after the definition of the menu for newapp:

         select USER_APPS_LIBAPP

in order to notify the SDK configuration system that in case newapp-1.0 is selected, then libapp-1.0 should be enabled too. We will notify that libapp-1.0 needs to be built before newapp-1.0 later on the metainfo file.

Now we will define the metainfo file for libapp-1.0 (and we will elaborate on it during the next sections):


And the metainfo file for newapp-1.0:


In the next sections you will learn what are those files are defining.

Providing dependency information for other components

When a target package (newapp) uses libraries (libapp) that can be shared with other target packages. It is necessary to provide location information on the libraries and their header files.

There are two procedures to consider:

The answer to this questions is the $DEVDIR/fs/Apps.defs file. This file is dynamically generated extracting meta information from each of the application packages metainfo files.

Looking at the libapp1.0 metainfo file we can see the following:


In the above lines, the NAME identifier is used to prefix the definitions that will be available in the $DEVDIR/fs/Apps.defs repository file.

Each one of the identifiers prefixed with DEFS_ represents a path to a specific component of the application, and they will appear in the $DEVDIR/fs/Apps.defs repository file when they need to be used by other applications. The DEFS_ prefix will be substituted with the content of the NAME identifier. In this way, each definition will be associated in a unique way with their correspondent application package. It is important that each application holds a unique NAME identifier.

Also, note that the DEFS_ strings includes the $(FSDEVROOT) substring, meaning that the headers and libraries are to be found on the /usr path of the file system for development.

Accordingly, the metainfo script will generate the definitions and the final output will appear in the $DEVDIR/fs/Apps.defs repository file looking like the following code block:


Now the Makefile of newapp-1.0 can include $(DEVDIR)/fs/Apps.defs and use the definitions to notify the compiler and linker about the proper location of headers and libraries of libapp-1.0.

Note: the previous SDK build system used an Apps.defs file for each application that included the required definitions just like the previous code block. The usage of those individual Apps.defs files are now deprecated. If the metainfo system founds both an Apps.defs and metainfo files in an application directory, the contents of the Apps.defs file will be used and those from the metainfo file will be discarded.

Detailing software dependencies

When a target application requires the availability of another specific target application or library, the dependency must be included in the TARGET_REQUIRED line. In our example, we found this line of the metainfo file of newapp-1.0:


The identifier representing each application is just the name of the required application's directory as found in the upper level directory of the application metainfo file requesting it. This identifier may be a white space separated list of directories.

The SDK build system will build those applications before the application that requires them, in order to comply with the requirement, in turn, if any required library or application holds other dependencies, they will be built before it and so on recursively.

Note: the previous SDK build systems, there was a file named dependencies that included each directory name in its own line. If a dependencies file exists, it will take precedence over the metainfo file.

Detailing Software dependencies on host hardware =

In the metainfo file, the line with the identifier HOST_REQUIRED provides a white space separated list of the host system packages that should be installed. In our example newapp-1.0 requires flex to properly build.


This will instruct the SDK build system to verify if the host Linux distribution has installed the proper software package that provides the requested tool. Since the SDK is designed to run on different Linux distributions, there is a dictionary to translate between the program name and the package that should be queried for the particular distribution.

If you are using the SDK in a supported distribution and machine you can skip this step and just manually go ahead and verify the package is installed on your machine. However if you intend to re-distribute, or re-install the SDK on different machines, you are better adding the host tool check.

The text database files for each distribution are located in $DEVDIR/bsp/oscheck/, and have been named with the format of distribution-version_architecturebits, such as ubuntu-6.06_32.

In those files there are defined a list of pairs, in the format <alias>:<package_name>. Example:


That is, each line contains an alias or symbolic identifier, a colon, and the name of the package in the host system distribution according to the package management tool used. The SDK has building support for querying the distribution package management tool.

Other helper functions of the metainfo file

The metainfo files can be used to support other more obscure operations.


Where the “USER_APPS_MYAPP” definition corresponds to the config flag that should be used to detect if the application is enabled or not.

Porting autotools based applications

One of the main reasons for using embedded Linux is the vast amount of open source libraries and programs available. RidgeRun has simplified the process of porting open source applications into the SDK.

Most open source applications available use what is known as the GNU build system, also called autotools. For more information on autotools, the following introductory documentation is recommended: http://www.lrde.epita.fr/~adl/dl/autotools.pdf .

Autotools generates scripts that in turn detect if the needed components are available in the build environment. For the SDK, that includes both the host components (toolchain, make, autoconf, shell scripts) and the target components (any library required to be available in the embedded target).

Most open source applications follow a basic three step build process based on autotools using the following order: 1. configure: Checks for required host and target components and the build system capabilities. 2. make: Builds the applications. 3. make install: Install the application in the target path.

Before getting into details of porting autotools based applications, please be sure that you have read and understood the section Integrating applications in the SDK.

Wrapper Makefile

In order to integrate autotools based applications into the SDK, it is necessary to write a wrapper makefile with the standard targets required by the SDK build system. The SDK system already includes a standard template that should work with most autotools based applications if they are properly constructed following the autotools guidelines. The SDK template Makefile that needs to be used is:

   # License header...
   .PHONY: build chkconfig preconfig buildfs install clean
   ifeq ($(DEVDIR),)
   $(error ====== DEVDIR variable is empty, invoke this Makefile from the
   BSP root, or provide the path to it =====)
   include $(DEVDIR)/bsp/mach/Make.conf
   include $(DEVDIR)/fs/Apps.defs

   build: configuration built
   built: configured
        $(V) $(MAKE) -C $(AUTOTOOLS_BUILD_DIR) $(QOUT)
        $(V) touch $@
        $(V) $(MAKE) -C $(AUTOTOOLS_BUILD_DIR) install $(QOUT)
        $(V) touch $@
   install: installed
        $(V) install -Dm 755 $(FSDEVROOT)/usr/bin/PortedApp
        -$(V) $(MAKE) -C $(AUTOTOOLS_BUILD_DIR) uninstall $(QOUT)
        -$(V) rm -f $(FSROOT)/usr/bin/PortApp
   clean: clean_configure
        $(V) rm -Rf *sim installed built
   include $(DEVDIR)/bsp/autotools.defs


NOTE: this is a SDK template and is expected that you change and adjust the minimal definitions regarding the application you are porting (application name and configuration name variable).

This SDK template makefile uses several definitions that are encapsulated on $(DEVDIR)/bsp/autotools.defs and $(DEVDIR)/bsp/config.site. These definitions will take care of invoking autotools to configure it properly for the target hardware.

The SDK makefile will compile and install the application under $(FSDEVROOT) which usually is set to $(DEVDIR)/fs/fsdev. The propose of this directory is to fully install the autotools based application there using the application install target and later to copy the minimal required files to the target file system on $(FSROOT). This is performed in this way since often the install target will add several MB of files that are not required for run-time execution (headers, localizations, manuals, etc), making the target file system bigger.

There are some aspects that you may customize of this scripts:

Please note that not all autotools based applications will compile without problems, and there are certain cases that are know to be specially problematic: