Prerequisites for building the OpenDataPlane (ODP) API 1. Linux CentOS 7 (kernel v3.10) is the oldest Linux distributions tested by the ODP CI. Earlier versions may or may not work. For CentOS/RedHat distros, configure the system to use Fedora EPEL repos and third-party packages: $ sudo yum install epel-release Additionally, for CentOS 8 distros, enable the powertools repository: $ sudo yum install dnf-plugins-core $ sudo yum config-manager --set-enabled powertools 2. autotools automake autoconf libtool pkg-config On Debian/Ubuntu systems: $ sudo apt-get install automake autoconf libtool pkg-config On CentOS/RedHat/Fedora systems: $ sudo yum install automake autoconf libtool pkgconfig 3. Required packages Libraries currently required to link: libconfig, openssl, libatomic On Debian/Ubuntu systems: $ sudo apt-get install libconfig-dev libatomic On CentOS/RedHat/Fedora systems: $ sudo yum install libconfig-devel libatomic It is possible to build ODP without OpenSSL by passing flag --without-openssl to configure script. However this will result in ODP-generic not supporting any cryptography algorithm (except NULL) and any strong random generation (only BASIC is supported and it is not cryptographically secure). Use at your own risk! 3.1 OpenSSL native compile For native compilation, simply load the necessary libraries using the appropriate tool set. On Debian/Ubuntu systems: $ sudo apt-get install libssl-dev On CentOS/RedHat/Fedora systems: $ sudo yum install openssl-devel 3.2 OpenSSL cross compilation Cross compilation requires cross compiling the individual libraries. In order for a cross compiled executable to run on a target system, one must build the same version as that which is installed on the target rootfs. For example, to build openssl for both 32 and 64 bit compilation: # Clone openssl repository $ git clone git://git.openssl.org/openssl.git $ cd openssl # The command "git tag" will list all tags available in the repo. $ git tag # Checkout the specific tag to match openssl library in your target rootfs $ git checkout # Build and install 32 bit version of openssl $ ./Configure linux-generic32 --cross-compile-prefix=arm-linux-gnueabihf- \ --prefix=/home/${USER}/src/install-openssl shared $ make $ make install # Build and install 64 bit version of openssl $ ./Configure linux-generic64 --cross-compile-prefix=aarch64-linux-gnu- \ --prefix=/home/${USER}/src/install-openssl-aarch64 shared $ make $ make install # You may now build either 32 or 64 bit ODP $ git clone https://github.com/OpenDataPlane/odp.git odp $ cd odp $ ./bootstrap # Build 32 bit version of ODP $ ./configure --host=arm-linux-gnueabihf \ --with-openssl-path=/home/${USER}/src/install-openssl $ make # Or build 64 bit version of ODP $ ./configure --host=aarch64-linux-gnu \ --with-openssl-path=/home/${USER}/src/install-openssl-aarch64 $ make 3.3 ARMv8 Crypto native compile (optional) AArch64cryptolib is a from-scratch implementation of cryptographic primitives aiming for optimized performance on Arm A-class cores. This implementation requires Armv8 cryptography extensions. Currently, only AES-GCM algorithm is supported. 3.3.1 Building ARMv8 crypto library # Checkout and build AArch64cryptolib code $ git clone https://github.com/ARM-software/AArch64cryptolib $ cd AArch64cryptolib/ $ make For additional instructions, refer to README.md 3.3.2 Building ODP with ARMv8 crypto library $ ./bootstrap # Append the Armv8 crypto library path to PKG_CONFIG_PATH if necessary $ echo $PKG_CONFIG_PATH $ export PKG_CONFIG_PATH=/pkgconfig:$PKG_CONFIG_PATH # Compile and build ODP with Armv8 crypto library $ ./configure --with-crypto=armv8crypto $ make 3.4 Multi-buffer Crypto for IPsec Library (optional) Multi-Buffer Crypto for IPsec Library is a set of functions that implement authentication and encryption processing for IPsec, these functions take advantage of SIMD instructions to improve performance. Note ODP assumes that IPSec MB library is compiled with SAFE_PARAM enabled (enabled by default), otherwise crypto operations with too long cipher/auth ranges will have undefined behaviour. 3.4.1 Building Multi-buffer Crypto for IPSec Library for Arm # Checkout and build Arm code $ git clone https://git.gitlab.arm.com/arm-reference-solutions/ipsec-mb.git $ cd ipsec-mb/ $ git checkout SECLIB-IPSEC-2022.12.13 $ make $ sudo make install For additional instructions, refer to README.md in crypto library repository. 3.4.2 Building ODP with Multi-buffer IPSec Library $ ./bootstrap # Compile and build ODP with Multi-buffer IPSec library $ ./configure --with-crypto=ipsecmb $ make 3.5 DPDK packet I/O support (optional) Use DPDK for ODP packet I/O. Currently supported DPDK versions are v19.11, v20.11, v21.11, and v22.11 (recommended). Note: only packet I/O is accelerated with DPDK. See https://github.com/OpenDataPlane/odp-dpdk.git for a full DPDK based ODP implementation. 3.5.1 DPDK pktio requirements DPDK pktio adds a dependency to NUMA library. # Debian/Ubuntu $ sudo apt-get install libnuma-dev # CentOS/RedHat/Fedora $ sudo yum install numactl-devel 3.5.2 Native DPDK install # Debian/Ubuntu starting from 20.04 $ sudo apt-get install dpdk-dev 3.5.3 Build DPDK v19.11 from source $ git clone https://dpdk.org/git/dpdk-stable --branch 19.11 --depth 1 ./ # Make and edit DPDK configuration $ export TARGET="x86_64-native-linuxapp-gcc" $ make config T=${TARGET} O=${TARGET} $ pushd ${TARGET} # Enable pcap PMD to use DPDK pktio without supported NICs: $ sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config $ popd # Build and install DPDK $ make install T=${TARGET} EXTRA_CFLAGS="-fPIC" DESTDIR=./install # Configure ODP $ ./configure --with-dpdk-path= 3.5.4 Build DPDK v20.11 and onwards from source $ git clone https://dpdk.org/git/dpdk-stable --branch --depth 1 ./ # Prepare the build directory $ cd $ meson build $ cd build # Optionally, configure the location where DPDK will be installed. By default, # DPDK will be installed in /usr/local: $ meson configure -Dprefix=$(pwd)/../install # Build and install DPDK $ ninja install # Configure ODP $ ./configure --enable-dpdk # Or, if DPDK was not installed to the default location, set PKG_CONFIG_PATH: $ PKG_CONFIG_PATH=/install/lib/x86_64-linux-gnu/pkgconfig ./configure --enable-dpdk 3.5.5 Setup system # Load DPDK modules $ sudo modprobe uio $ cd $ sudo insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko Reserve and mount huge pages and bind supported interfaces to DPDK modules following the DPDK documentation. ODP DPDK packet I/O has been tested with 512 x 2MB huge pages. All this can be done with the DPDK setup script (/usertools/dpdk-setup.sh). 3.5.6 Running ODP with DPDK pktio ODP applications will try use DPDK for packet I/O by default. If some other I/O type is desired instead, DPDK I/O can be disabled by setting the environment variable ODP_PKTIO_DISABLE_DPDK. DPDK interfaces are accessed using indexes. For example, two first DPDK interfaces can be used with the odp_l2fwd example as follows: $ cd $ sudo ./test/performance/odp_l2fwd -i 0,1 -c 2 -m 0 Additionally, DPDK command line options can be passed to the application using ODP_PKTIO_DPDK_PARAMS environment variable. For example, allocate 1024MB of memory: $ sudo ODP_PKTIO_DPDK_PARAMS="-m 1024" ./test/performance/odp_l2fwd -i 0 -c 1 3.6 AF_XDP socket based packet I/O support (optional) Use AF_XDP socket for packet I/O. A kernel version of 5.10 or higher is required, older kernels may or may not work. More information about XDP and AF_XDP can be found here: https://www.kernel.org/doc/Documentation/networking/af_xdp.rst The status of the implementation is **experimental** and may cause issues e.g. with some packet length, packet segment length and pool size combinations that would otherwise conform to reported capabilities. Note that, currently, AF_XDP socket packet I/O cannot be instantiated if DPDK zero-copy is enabled. Additionally, RSS hash key and flow hash configuration is done based on the NIC/driver default values and should be manually reconfigured with e.g. ethtool if changes are required. 3.6.1 AF_XDP socket packet I/O requirements AF_XDP socket packet I/O implementation requires libxdp and libbpf libraries. They can be fetched from XDP-project in GitHub: $ git clone https://github.com/xdp-project/xdp-tools (Contains submodules which should be cloned as well.) Additional packages might be needed to be installed as well: llvm-dev and gcc-multilib. $ ./configure $ make After building, libraries should be installed. $ cd $ make install $ cd $ make install 3.6.2 Build ODP with AF_XDP socket packet I/O support After building and installing libxdp and libbpf, ODP can be configured to be built with AF_XDP support (modify PKG_CONFIG_PATH as needed). $ ./configure --enable-xdp 3.6.3 Running ODP with AF_XDP socket packet I/O with Mellanox NICs AF_XDP socket packet I/Os bind to NIC TRX-combined queues. Steering packets to correct input sockets is configured via NIC RSS. With some NICs (e.g. Mellanox), the driver queue configuration is adjusted by the NIC with additional queues on top of the configured amount of TRX queues which are then used as the actual queues. This will require additional forwarding rules as RSS is not possible to be configured programmatically in this case. Otherwise packets do not get forwarded to the correct queues. For example: ethtool -N flow-type ether dst action 5 Would forward Ethernet frames with a given destination address to queue 5, which would be the first actual queue in case 5 regular combined queues were configured (zero-indexing). 3.7 Machine Learning API support (optional) Use ML API for model inferencing. ML implementation uses ONNX Runtime library (https://github.com/microsoft/onnxruntime). ODP has been tested with ONNX Runtime version 1.16.3. 3.7.1 Prebuilt onnxruntime download Download a default CPU version onnxruntime-linux-x64-*.tgz and unzip it to any folder. $ wget -P ~ https://github.com/microsoft/onnxruntime/releases/download/v/onnxruntime-linux-x64-.tgz $ mkdir $ cd / $ tar --strip=1 -zxvf ~/onnxruntime-linux-x64-.tgz 3.7.1 Build onnxruntime from source $ git clone --recursive https://github.com/Microsoft/onnxruntime.git $ cd onnxruntime # Configure $ ./build.sh --config RelWithDebInfo --build_shared_lib --parallel $ tools/ci_build/github/linux/copy_strip_binary.sh -r build/Linux/ -a onnxruntime -l libonnxruntime.so.1.14.0 -c RelWithDebInfo -s . -t $ cp -r build/Linux/onnxruntime/ 3.7.2 Build ODP with ML support After installing onnxruntime and example dependencies, ODP can be configured to be built with ML support by giving onnxruntime path with --with-ort-path. $ ../configure --with-ort-path= 4.0 Packages needed to build API tests CUnit test framework version 2.1-3 is required CUnit provides a framework to run the API test suite that proves conformance to the ODP API. The home page http://cunit.sourceforge.net/doc/introduction.html 4.1 Native CUnit install # Debian/Ubuntu $ sudo apt-get install libcunit1-dev # CentOS/RedHat/Fedora systems $ sudo yum install CUnit-devel 4.2 Built from src export CUNIT_VERSION=2.1-3 curl -sSOL http://sourceforge.net/projects/cunit/files/CUnit/${CUNIT_VERSION}/CUnit-${CUNIT_VERSION}.tar.bz2 tar -jxf *.bz2 cd CUnit* ./bootstrap # Install CUnit into the default location (/usr/local). This is needed for # 'make distcheck'. ./configure make sudo make install # On RedHat you also have to add path /usr/local/lib to /etc/ld.so.conf # ... OR ... Install CUnit into user defined location. The same path is # used in step 4.4: PKG_CONFIG_PATH=/home//lib/pkgconfig ./configure --prefix=/home/ make make install # Also (in Ubuntu/RedHat at least) run ldconfig to update shared lib # cache or reboot, before trying to run e.g. 'make distcheck'. sudo ldconfig 4.3 Cross compile of CUnit $ git svn clone http://svn.code.sf.net/p/cunit/code/trunk cunit-code $ cd cunit-code $ ./bootstrap # Build and install 32 bit version of cunit $ ./configure --host=arm-linux-gnueabihf --prefix=/home/${USER}/src/install-cunit $ make $ make install # Build and install 64 bit version of cunit $ ./configure --host=aarch64-linux-gnu --prefix=/home/${USER}/src/install-cunit $ make $ make install 4.4 Using CUnit with ODP Configure will automatically look for CUnit if validation testsuite is enabled. By default it uses pkg-config to locate CUnit. Usually no additional configuration will be required. Few corner cases: # User directory installation ./configure PKG_CONFIG_PATH=/home//lib/pkgconfig # ... OR directly specifying flags ./configure CUNIT_CFLAGS="-I/home//include" CUNIT_LIBS="/home//lib -lcunit" 5.0 Cross Compilation for arm64 To cross compile binaries for arm64 on an x86_64 Debian based system, install the following packages: crossbuild-essential-arm64, libconfig-dev:arm64 (optionally libssl-dev:arm64). 5.1 Pre-installation setup (optional) # Debian/Ubuntu $ sudo dpkg --add-architecture arm64 Modify /etc/apt/sources.list to add remote repositories for fetching arm64 software packages for your Ubuntu version. Once this is complete, run: $ sudo apt-get update 5.2 Install packages # Debian/Ubuntu $ sudo apt-get install crossbuild-essential-arm64 $ sudo apt-get install libconfig-dev:arm64 Installing OpenSSL is optional. Refer to section 3 for more details. $ sudo apt-get install libssl-dev:arm64 5.3 Building ODP $ ./bootstrap $ ./configure --host=aarch64-linux-gnu $ make To build ODP with cross-compiled cunit for arm64, refer to sections 4.3 and 4.4. 6.0 Documentation Images & Doxygen Images are stored as svg files. No conversions for these are needed. Message sequence diagrams are stored as msc files and the svg versions are generated when the docs are built. # Debian/Ubuntu $ sudo apt-get install mscgen # CentOS 8/RedHat/Fedora $ sudo yum install mscgen 6.1 API Guide See https://www.doxygen.nl/manual/install.html The tested version of doxygen is 1.8.8 6.1.1 HTML # Debian/Ubuntu $ sudo apt-get install doxygen graphviz # CentOS/RedHat/Fedora $ sudo yum install doxygen graphviz 6.2 User guides 6.2.1 HTML # Debian/Ubuntu $ sudo apt-get install asciidoctor source-highlight librsvg2-bin # CentOS/RedHat/Fedora $ sudo yum install asciidoc source-highlight librsvg2 7.0 Submitting patches When submitting patches they should be checked with ./scripts/checkpatch.pl To have this tool also check spelling you need codespell. # Debian/Ubuntu $ sudo apt-get install codespell 8.0 Command Line Interface (optional) ODP applications (see e.g. ./example/cli) may use CLI helper (./helper/include/odp/helper/cli.h) to provide a command line interface to the user. CLI helper depends on libcli library. # Debian/Ubuntu $ sudo apt-get install libcli-dev # CentOS/RedHat/Fedora $ sudo yum install libcli-devel