28. Development Kit Build System

The DPDK requires a build system for compilation activities and so on. This section describes the constraints and the mechanisms used in the DPDK framework.

There are two use-cases for the framework:

  • Compilation of the DPDK libraries and sample applications; the framework generates specific binary libraries, include files and sample applications
  • Compilation of an external application or library, using an installed binary DPDK

28.1. Building the Development Kit Binary

The following provides details on how to build the DPDK binary.

28.1.1. Build Directory Concept

After installation, a build directory structure is created. Each build directory contains include files, libraries, and applications.

A build directory is specific to a configuration that includes architecture + execution environment + toolchain. It is possible to have several build directories sharing the same sources with different configurations.

For instance, to create a new build directory called my_sdk_build_dir using the default configuration template config/defconfig_x86_64-linuxapp, we use:

cd ${RTE_SDK}
make config T=x86_64-native-linuxapp-gcc O=my_sdk_build_dir

This creates a new my_sdk_build_dir directory. After that, we can compile by doing:

cd my_sdk_build_dir
make

which is equivalent to:

make O=my_sdk_build_dir

The content of the my_sdk_build_dir is then:

-- .config                         # used configuration

-- Makefile                        # wrapper that calls head Makefile
                                   # with $PWD as build directory


    -- build                              #All temporary files used during build
    +--app                                # process, including . o, .d, and .cmd files.
        |  +-- test                       # For libraries, we have the .a file.
        |  +-- test.o                     # For applications, we have the elf file.
        |  `-- ...
        +-- lib
            +-- librte_eal
            |   `-- ...
            +-- librte_mempool
            |  +--  mempool-file1.o
            |  +--  .mempool-file1.o.cmd
            |  +--  .mempool-file1.o.d
            |  +--   mempool-file2.o
            |  +--  .mempool-file2.o.cmd
            |  +--  .mempool-file2.o.d
            |  `--  mempool.a
            `-- ...

-- include                # All include files installed by libraries
    +-- librte_mempool.h  # and applications are located in this
    +-- rte_eal.h         # directory. The installed files can depend
    +-- rte_spinlock.h    # on configuration if needed (environment,
    +-- rte_atomic.h      # architecture, ..)
    `-- \*.h ...

-- lib                    # all compiled libraries are copied in this
    +-- librte_eal.a      # directory
    +-- librte_mempool.a
    `-- \*.a ...

-- app                    # All compiled applications are installed
+ --test                  # here. It includes the binary in elf format

Refer to Development Kit Root Makefile Help for details about make commands that can be used from the root of DPDK.

28.2. Building External Applications

Since DPDK is in essence a development kit, the first objective of end users will be to create an application using this SDK. To compile an application, the user must set the RTE_SDK and RTE_TARGET environment variables.

export RTE_SDK=/opt/DPDK
export RTE_TARGET=x86_64-native-linuxapp-gcc
cd /path/to/my_app

For a new application, the user must create their own Makefile that includes some .mk files, such as ${RTE_SDK}/mk/rte.vars.mk, and ${RTE_SDK}/mk/ rte.app.mk. This is described in Building Your Own Application.

Depending on the chosen target (architecture, machine, executive environment, toolchain) defined in the Makefile or as an environment variable, the applications and libraries will compile using the appropriate .h files and will link with the appropriate .a files. These files are located in ${RTE_SDK}/arch-machine-execenv-toolchain, which is referenced internally by ${RTE_BIN_SDK}.

To compile their application, the user just has to call make. The compilation result will be located in /path/to/my_app/build directory.

Sample applications are provided in the examples directory.

28.3. Makefile Description

28.3.1. General Rules For DPDK Makefiles

In the DPDK, Makefiles always follow the same scheme:

  1. Include $(RTE_SDK)/mk/rte.vars.mk at the beginning.

  2. Define specific variables for RTE build system.

  3. Include a specific $(RTE_SDK)/mk/rte.XYZ.mk, where XYZ can be app, lib, extapp, extlib, obj, gnuconfigure, and so on, depending on what kind of object you want to build. See Makefile Types below.

  4. Include user-defined rules and variables.

    The following is a very simple example of an external application Makefile:

    include $(RTE_SDK)/mk/rte.vars.mk

# binary name
APP = helloworld

# all source are stored in SRCS-y
SRCS-y := main.c

CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)

include $(RTE_SDK)/mk/rte.extapp.mk

28.3.2. Makefile Types

Depending on the .mk file which is included at the end of the user Makefile, the Makefile will have a different role. Note that it is not possible to build a library and an application in the same Makefile. For that, the user must create two separate Makefiles, possibly in two different directories.

In any case, the rte.vars.mk file must be included in the user Makefile as soon as possible.

28.3.2.1. Application

These Makefiles generate a binary application.

  • rte.app.mk: Application in the development kit framework
  • rte.extapp.mk: External application
  • rte.hostapp.mk: prerequisite tool to build dpdk

28.3.2.2. Library

Generate a .a library.

  • rte.lib.mk: Library in the development kit framework
  • rte.extlib.mk: external library
  • rte.hostlib.mk: host library in the development kit framework

28.3.2.3. Install

  • rte.install.mk: Does not build anything, it is only used to create links or copy files to the installation directory. This is useful for including files in the development kit framework.

28.3.2.4. Kernel Module

  • rte.module.mk: Build a kernel module in the development kit framework.

28.3.2.5. Objects

  • rte.obj.mk: Object aggregation (merge several .o in one) in the development kit framework.
  • rte.extobj.mk: Object aggregation (merge several .o in one) outside the development kit framework.

28.3.2.6. Misc

  • rte.doc.mk: Documentation in the development kit framework
  • rte.gnuconfigure.mk: Build an application that is configure-based.
  • rte.subdir.mk: Build several directories in the development kit framework.

28.3.3. Internally Generated Build Tools

app/dpdk-pmdinfogen

dpdk-pmdinfogen scans an object (.o) file for various well known symbol names. These well known symbol names are defined by various macros and used to export important information about hardware support and usage for pmd files. For instance the macro:

RTE_PMD_REGISTER_PCI(name, drv)

Creates the following symbol:

static char this_pmd_name0[] __attribute__((used)) = "<name>";

Which dpdk-pmdinfogen scans for. Using this information other relevant bits of data can be exported from the object file and used to produce a hardware support description, that dpdk-pmdinfogen then encodes into a json formatted string in the following format:

static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";

These strings can then be searched for by external tools to determine the hardware support of a given library or application.

28.3.4. Useful Variables Provided by the Build System

  • RTE_SDK: The absolute path to the DPDK sources. When compiling the development kit, this variable is automatically set by the framework. It has to be defined by the user as an environment variable if compiling an external application.
  • RTE_SRCDIR: The path to the root of the sources. When compiling the development kit, RTE_SRCDIR = RTE_SDK. When compiling an external application, the variable points to the root of external application sources.
  • RTE_OUTPUT: The path to which output files are written. Typically, it is $(RTE_SRCDIR)/build, but it can be overridden by the O= option in the make command line.
  • RTE_TARGET: A string identifying the target for which we are building. The format is arch-machine-execenv-toolchain. When compiling the SDK, the target is deduced by the build system from the configuration (.config). When building an external application, it must be specified by the user in the Makefile or as an environment variable.
  • RTE_SDK_BIN: References $(RTE_SDK)/$(RTE_TARGET).
  • RTE_ARCH: Defines the architecture (i686, x86_64). It is the same value as CONFIG_RTE_ARCH but without the double-quotes around the string.
  • RTE_MACHINE: Defines the machine. It is the same value as CONFIG_RTE_MACHINE but without the double-quotes around the string.
  • RTE_TOOLCHAIN: Defines the toolchain (gcc , icc). It is the same value as CONFIG_RTE_TOOLCHAIN but without the double-quotes around the string.
  • RTE_EXEC_ENV: Defines the executive environment (linuxapp). It is the same value as CONFIG_RTE_EXEC_ENV but without the double-quotes around the string.
  • RTE_KERNELDIR: This variable contains the absolute path to the kernel sources that will be used to compile the kernel modules. The kernel headers must be the same as the ones that will be used on the target machine (the machine that will run the application). By default, the variable is set to /lib/modules/$(shell uname -r)/build, which is correct when the target machine is also the build machine.
  • RTE_DEVEL_BUILD: Stricter options (stop on warning). It defaults to y in a git tree.

28.3.5. Variables that Can be Set/Overridden in a Makefile Only

  • VPATH: The path list that the build system will search for sources. By default, RTE_SRCDIR will be included in VPATH.
  • CFLAGS: Flags to use for C compilation. The user should use += to append data in this variable.
  • LDFLAGS: Flags to use for linking. The user should use += to append data in this variable.
  • ASFLAGS: Flags to use for assembly. The user should use += to append data in this variable.
  • CPPFLAGS: Flags to use to give flags to C preprocessor (only useful when assembling .S files). The user should use += to append data in this variable.
  • LDLIBS: In an application, the list of libraries to link with (for example, -L /path/to/libfoo -lfoo ). The user should use += to append data in this variable.
  • SRC-y: A list of source files (.c, .S, or .o if the source is a binary) in case of application, library or object Makefiles. The sources must be available from VPATH.
  • INSTALL-y-$(INSTPATH): A list of files to be installed in $(INSTPATH). The files must be available from VPATH and will be copied in $(RTE_OUTPUT)/$(INSTPATH). Can be used in almost any RTE Makefile.
  • SYMLINK-y-$(INSTPATH): A list of files to be installed in $(INSTPATH). The files must be available from VPATH and will be linked (symbolically) in $(RTE_OUTPUT)/$(INSTPATH). This variable can be used in almost any DPDK Makefile.
  • PREBUILD: A list of prerequisite actions to be taken before building. The user should use += to append data in this variable.
  • POSTBUILD: A list of actions to be taken after the main build. The user should use += to append data in this variable.
  • PREINSTALL: A list of prerequisite actions to be taken before installing. The user should use += to append data in this variable.
  • POSTINSTALL: A list of actions to be taken after installing. The user should use += to append data in this variable.
  • PRECLEAN: A list of prerequisite actions to be taken before cleaning. The user should use += to append data in this variable.
  • POSTCLEAN: A list of actions to be taken after cleaning. The user should use += to append data in this variable.
  • DEPDIR-y: Only used in the development kit framework to specify if the build of the current directory depends on build of another one. This is needed to support parallel builds correctly.

28.3.6. Variables that can be Set/Overridden by the User on the Command Line Only

Some variables can be used to configure the build system behavior. They are documented in Development Kit Root Makefile Help and External Application/Library Makefile Help

  • WERROR_CFLAGS: By default, this is set to a specific value that depends on the compiler. Users are encouraged to use this variable as follows:

    CFLAGS += $(WERROR_CFLAGS)

This avoids the use of different cases depending on the compiler (icc or gcc). Also, this variable can be overridden from the command line, which allows bypassing of the flags for testing purposes.

28.3.7. Variables that Can be Set/Overridden by the User in a Makefile or Command Line

  • CFLAGS_my_file.o: Specific flags to add for C compilation of my_file.c.
  • LDFLAGS_my_app: Specific flags to add when linking my_app.
  • EXTRA_CFLAGS: The content of this variable is appended after CFLAGS when compiling.
  • EXTRA_LDFLAGS: The content of this variable is appended after LDFLAGS when linking.
  • EXTRA_LDLIBS: The content of this variable is appended after LDLIBS when linking.
  • EXTRA_ASFLAGS: The content of this variable is appended after ASFLAGS when assembling.
  • EXTRA_CPPFLAGS: The content of this variable is appended after CPPFLAGS when using a C preprocessor on assembly files.