path: root/doc/implementers-guide/implementers-guide.adoc

:doctitle: OpenDataPlane (ODP) Implementers-Guide
:description: This document is intended to guide a new OpenDataPlane +
implementation developer.
:imagesdir: ../images
:toc:

:numbered!:
[abstract]
Abstract
--------
This document is intended to guide a new ODP implementation developer.
Further details about ODP may be found at http://opendataplane.org[ODP homepage]

:numbered:

== Introduction and Overview
ODP consists of three distinct components:

[[odp_components]]
.OpenDataPlane Components
image::odp_components.svg[align="center"]

- An abstract API specification. This is the application's writer's view of
ODP and defines the syntax and semantics of all ODP routines at a functional
level.

- An Implementation of the ODP API for a specific target platform. A
mapping of the ODP APIs to a specific target platform. This is the focus of
this document.

- A Validation Test Suite. This is an independent set of routines that when
run against an ODP implementation verifies that it correctly implements all of
the defined ODP APIs at a functional level. The test suite is used by
implementers to self-certify their ODP implementation as well as by
third-parties to verify an implementation's claim to be ODP API compliant.

The structure of these components, especially the API specification, is more
fully defined in the _ODP User's Guide_

=== Organization of this Document
This document is designed to serve two purposes. Its larger purpose is to
provide guidance and practical advice for those wishing to implement ODP on
their platform. To help with this, as well as to provide deeper insight into
how to think about ODP from an implementer's standpoint, this document also
discusses in some depth the design and organization of a specific ODP
implementation: the odp-linux reference implementation distributed as part of
the main ODP git repository. By grounding theory in practice and discussing
a particular example implementation, it is hoped this will provide insight into
the trade-offs implementers should consider in approaching how to best implement
ODP on their platforms.

The section <<include_structure>> discusses the layout of the ODP include tree
from an implementer's perspective. Although implementers have wide latitude
in how they organize their ODP implementations, it is recommended that this
layout be be observed by other implementations. Doing so both simplifies code
sharing with the odp-linux reference implementation and also ensure ease of
upgrading to future ODP API levels, as this is the basic layout that will be
observed by future revisions to the API. This layout also facilitates shared
library distribution.

The section <<validation>> then discusses how validation tests are organized
and run to provide ODP API conformance testing for ODP implementations. This
is something ODP implementations need to consider from the outset. ODP uses
a self-certifying model for API compliance, however this self-certification is
performed by the ODP validation test suite (possibly augmented with a vendor's
own extensions to this suite). The important point is that because the
validation suite is itself free open source code, any potential user of a
given ODP implementation is free to do its own validation of a vendor's ODP
implementation using this suite.

Following this basic material, each ODP API area is then reviewed and its
implementation considerations are discussed, illustrating these considerations
with discussion of how these are done in *odp-linux*.

[[include_structure]]
== The Include Structure

The implementers view of the include source tree allows the common interface
definitions and documentation to be reused by all the platforms defined in the
tree, but leave the actual definitions to be defined by the specific platform.
The different ODP interfaces (api and drv) are defined and implemented using
similar structures:

.Implementers include structure (in repository)
----
./
├── include/
│   ├── odp/
│   │   ├── api/
│   │   │   └── spec/
│   │   │       └── The Public API specification and its documentation. <1>
│   │   │
│   │   └── drv/
│   │       └── spec/
│   │           └── The Public Nic driver interface and its documentation. <5>
│   │
│   │
│   ├── odp_api.h  This file should be the only file included by the any ODP
│   │              application. <4>
│   │
│   └── odp_drv.h  This file should be the only file included by the any ODP
│                  nic driver. <8>
│
└── platform/
    └── <implementation name>/
	└── include/
	    ├── Internal header files seen only by the implementation.
	    └── odp/
		├── api/ <2>
		│   ├── In-line function definitions of the public API for this
		│   │   platform seen by the application.
		│   │
		│   └── plat/ <3>
		│       └── Platform specific types, enums etc as seen by the
		│           application but require overriding by the
		│           implementation.
		│
		├── drv/ <6>
		│   ├── In-line function definitions of the nic driver interface
		│   │   for this platform seen by the application.
		│   │
		│   └── plat/ <7>
		│       └── Platform specific types, enums etc as seen by the
		│           nic driver but require overriding by the
		│           implementation.
		│
		└── com/
		    └── Things common to both interfaces are placed here.

----
<1> The specification, defining the ODP application programming interface (API)
is held in 'include/odp/api/spec/'. The ODP API is defined by a set of '.h'
files including doxygen documentation.
<2> Each public specification file is included by a counterpart in
'platform/<implementation name>/include/odp/api'.
The include of the specification API is AFTER the platform specific definitions
to allow the platform to provide definitions that match the underlying hardware.
<3> The implementation code may include files from
'platform/<implementation name>/include/odp/api/plat'
<4> Applications in turn include the include/odp_api.h file which includes the
'platform/<implementation name>/include/odp/api' files to provide a complete
definition of the API.
<5> The specification, defining the driver programming interface (drv)
is held in 'include/odp/drv/spec/'. The interface is defined by a set of '.h'
files including doxygen documentation.
<6> Each public specification file is included by a counterpart in
'platform/<implementation name>/include/odp/drv'.
The include of the specification is AFTER the platform specific definitions
to allow the platform to provide definitions that match the underlying hardware.
<7> The implementation code may include files from
'platform/<implementation name>/include/odp/drv/plat'
<8> Nic drivers in turn include the include/odp_drv.h file which includes the
'platform/<implementation name>/include/odp/drv' files to provide a complete
definition of the ODP driver interface.



After ODP installation (make install), the structure becomes as follows:

.Installed ODP structure
----
./
└── include/
    ├── odp/
    │   ├── api/      API In-line for this platform.
    │   │   ├── plat/ API Platform specific types.
    │   │   └── spec/ The public API specification.
    │   │
    │   └── drv/      Driver interface In-line for this platform.
    │       ├── plat/ Driver interface Platform specific types.
    │       └── spec/ The public Driver interface specification.
    │
    ├── odp_api.h
    └── odp_drv.h
----

== ODP library naming recommendations

The ODP project supports two implementations *odp-linux* and *odp-dpdk*.  The
name of the libraries are `libodp-linux` and `libodp-dpdk` respectively. It is
recommended that other implementations follow the same schema
(*odp-<implementation name>*) to make the representation of the ODP
implementations uniform in a distribution.

[[validation]]
== The Validation Suite
ODP provides a comprehensive set of API validation tests that are intended to be
used by implementers during development and by application developers to verify
that a particular implementation meets their requirements.

The list of these tests is expected to grow as ODP grows.

Note that validation tests are not typically written by implementers, however
their structure and operation needs to be understood so that implementations
can properly run them to validate that they conform to the ODP API
specification. The only exception to this is platform-specific tests, as
described in <<platform-specific-tests>>. These may be written by platforms to
leverage the CUnit framework for their own internal test needs or to extend
the platform-agnostic tests with platform-specific logic.

The list of test executables is run by the automake test harness, when running
`make check`.
Therefore, as required by this harness, each executable should return 0 on
success (tests passed), 77 on inconclusive, or any other values on failure.
The automake functionality shows a status line (PASSED/FAIL...) for each of the
ran test executables.

It is expected that ODP developers will need to run tests as early as possible
in the development cycle, before all APIs have been implemented.
Besides, although there are no APIs that are formally listed as optional, it is
also expected that there may be cases where a subset of APIs remain
unimplemented on a particular platform.
Moreover, some platforms may require specific initialization/termination code
prior/after running the standard tests.

To accommodate with these platform disparities, the ODP validation has been
divided in two distinct areas:

* The platform agnostic area,
* A platform dependent area (one per platform).

=== Platform agnostic

This grouping defines tests that are expected to be executable and succeed on
any platform, though possibly with very different performance, depending on
the underlying platform.  They are written in plain C code, and may only use
functions defined in the standard libC (C99) library (besides the ODP
functions being tested, of course).  A free C99 specification can be found at
the http://www.open-std.org/JTC1/sc22/wg14/www/docs/n1256.pdf[open-std.org]
web site. No other languages (like scripting) are allowed as their usage
would make assumptions on the platform capability.

This area is located at: `test/common_plat/`

In this directory, tests are grouped by category:

* validation : groups of test defining the ODP compliance
* performance : tests checking system responsiveness
* miscellaneous

Each ODP interface contains modules, where each module groups the set of ODP
functions related to the same "topic" for the given interface.
Examples of modules for the application interface includes "classification"
(API functions dealing with ingress packets classification), time
(functions dealing with time, excluding timers which have their own module),
timer, ...
The complete module list can be seen at:
http://docs.opendataplane.org/master/linux-generic-doxygen-html/modules.html[ODP Modules] +
Within the platform agnostic area, the validation tests for a given interface
are also grouped by modules, matching the ODP interface modules: for instance,
`test/common_plat/validation/api` mainly contains a list of
directories matching each module name (as defined by the doxygen `@defgroup`
or `@ingroup` statement present in each API `.h` file).

Within each of these directories, a library (called `libtest<module>.la`) and
its associated `.h` file (called `<module>.h`) defines all the test functions
for this module as well as few other functions to initialize, terminate, and
group the tests.  An executable called `<module>_main*`, is also built. It is
permissible to generate more than one executable to cover the functionality in
the test library for the module.  These executable(s) shall call all the tests
for this module. See <<anchor-1, Module test and naming convention>> for
more details.

It is important to be aware that the tests defined for a given module
(defined in `test/common_plat/validation/api/<module>`) are focused to test
the ODP functions belonging to this module, but are not limited to use this
module's ODP functions only: many modules needs some interaction with some
other module to be tested.
The obvious illustration of this is for module "init" whose functions are
required by all tests of all other modules (as ODP needs to be initialized to
test anything else).

There is a `Makefile.am` located at the top of the platform agnostic area. Its
role is limited to the construction of the different test libraries and the
`<module>_main*` executables. No tests are run from this area when `make check`
is performed.

==== CUnit

Within a given test executable CUnit is used to run the different tests. The
usage of CUnit implies the following structure:

* Tests are simple C functions.
* Tests are grouped in arrays called test suites. Each test suite can be
associated with a suite initialization/termination function(s), called by CUnit
before and after the whole suite is run.
* An array of test suites (and associated init/term functions) defines the test
registry run by the test executable.

Moreover, two extra functions can be used to initialize/terminate the test
executable (these are not part of CUnit).
A test executable return success (0) if every test of each suite succeed.

More details about CUnit can be found in the
http://cunit.sourceforge.net/doc/index.html[CUnit User's Guide]

[[anchor-1]]
==== Module test and naming convention

* Tests, i.e. C functions which are used in CUnit test suites are named:
   *<Module>_test_+++*+++* where the suffix identifies the test.

* Test arrays, _i.e.,_ arrays of `odp_testinfo_t`, listing the test functions
  belonging to a suite, are called: *<Module>_suite+++[_*]+++* where the
  possible suffix can be used if many suites are declared.

* CUnit suite init and termination functions are called:
   *<Module>+++_suite[_*]_init()+++* and *<Module>+++_suite[_*]_term()+++*
   respectively, where the possible extra middle pattern can be used if
   many suites are declared.

* Suite arrays, _i.e.,_ arrays of `odp_suiteinfo_t` used in executables (CUnit
  registry) are called *<Module>+++_suites[_*]+++* where the possible suffix
  identifies the executable using it, if any.

* Main executable function(s), are called <Module>+++_main[_*]+++* where the
   *possible suffix identifies the executable, if any, using it.

* Init/term functions for the whole executable are called *<Module>_init* and
   *<Module>_term*

All the above symbols are part of the generated `libtest<Module>.la` libraries.
The generated main executable(s) (named `<module>_+++main[_*]+++`, where the
optional suffix is used to distinguish the executables belonging to the same
module, if many) simply call(s) the related `<Module>_main+++[_*]+++` from the
library.

=== Platform specific

These tests are located under `platform/<platform_name>/`. There is one such
area for each platform implementing ODP. This location will be referred as
<PLATFORM_SPECIFIC> in the rest of this document.

==== The normal case

If the considered platform needs no platform specific tests, this directory
simply needs to contain a single `Makefile.am` listing each of the executables
(named <module>_main) built from the platform agnostic area. The executables are
listed in the automake TEST variable and will therefore be run on "make check".

For the linux-generic platform, most tested modules fall into this category:
currently, the `test/linux-generic/test/Makefile.am` looks
roughly as follows:

[source,am]
----
include $(top_srcdir)/test/Makefile.inc
TESTS_ENVIRONMENT += TEST_DIR=${top_builddir}/test/common_plat/validation

ALL_API_VALIDATION = ${top_builddir}/test/common_plat/validation/api

SUBDIRS =

if test_vald
TESTS = validation/api/pktio/pktio_run.sh \
	validation/api/pktio/pktio_run_tap.sh \
	$(ALL_API_VALIDATION)/atomic/atomic_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/barrier/barrier_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/buffer/buffer_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/classification/classification_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/cpumask/cpumask_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/crypto/crypto_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/errno/errno_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/hash/hash_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/init/init_main_ok$(EXEEXT) \
	$(ALL_API_VALIDATION)/init/init_main_abort$(EXEEXT) \
	$(ALL_API_VALIDATION)/init/init_main_log$(EXEEXT) \
	$(ALL_API_VALIDATION)/lock/lock_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/packet/packet_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/pool/pool_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/queue/queue_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/random/random_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/scheduler/scheduler_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/std_clib/std_clib_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/thread/thread_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/time/time_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/timer/timer_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/traffic_mngr/traffic_mngr_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/shmem/shmem_main$(EXEEXT) \
	$(ALL_API_VALIDATION)/system/system_main$(EXEEXT) \


----

With the exception for module pktio, all other modules testing just involves
calling the platform agnostic <module>_main executables (in
/test/common_plat/validation/api).

==== Using other languages

The pktio module, above, is actually tested using a bash script. This script is
needed to set up the interfaces used by the tests. The `pktio_run` script
eventually calls the platform agnostic
`test/common_plat/validation/api/pktio/pktio_main` after setting up the
interfaces needed by the tests.
Notice that the path to the script, `validation/api/pktio/pktio_run.sh`,
is pointing to a file within the <PLATFORM_SPECIFIC> tree so is private to
this platform. Any languages supported by the tested platform can be used
there, as it will not impact other platforms.
The platform "private" executables (such as this script), of course, must also
return one of the return code expected by the automake test harness
(0 for success, 77 for skipped, other values for errors).

==== Defining test wrappers

The pktio case above is actually using a script as wrapper around the "standard"
(platform independent) test executable. Wrappers can also be defined by using
the `LOG_COMPILER` variable of automake.
This is applicable in cases where the same wrapper should be used for more then
one test, as the test name is passed has parameter to the wrapper. A wrapper is
just a program expecting one argument: the test name.

Automake also supports the usage different wrappers based of the executable
filename suffix. See
https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness]
for more information.

To add a wrapper around the executed test, just add the following `LOG_COMPILER`
definition line in the `<PLATFORM_SPECIFIC>/Makefile.am`:

[source,am]
----
...
if test_vald
LOG_COMPILER = $(top_srcdir)/platform/linux-generic/test/wrapper-script
TESTS = pktio/pktio_run \
...
----

Here follows a dummy example of what wrapper-script could be:

[source,bash]
----
#!/bin/bash

# The parameter, $1, is the name of the test executable to run
echo "WRAPPER!!!"
echo "running $1!"

# run the test:
$1
# remember the test result:
res=$?

echo "Do something to clean up the mess here :-)"
# return the test result.
exit $res
----

Note how the above script stores the return code of the test executable to
return it properly to the automake test harness.

[[platform-specific-tests]]
==== Defining platform specific tests

Sometimes, it may be necessary to call platform specific system calls to check
some functionality: For instance, testing `odp_cpumask_*` could involve checking
the underlying system CPU mask. On linux, such a test would require using the
CPU_ISSET macro, which is linux specific. Such a test would be written in
`<PLATFORM_SPECIFIC>/<test-group>/<interface>/cpumask/...` The contents of
this directory would be very similar to the contents of the platform agnostic
side cpu_mask tests (including a `Makefile.am`...), but platform specific test
would be written there.
`<PLATFORM_SPECIFIC>/Makefile.am` would then trigger the building of the
platform specific tests (by listing their module name in `SUBDIRS` and therefore
calling the appropriate `Makefile.am`) and then it would call both the platform
agnostic executable(s) and the platform specific test executable.

The shm module of the linux-generic ODP API does have a validation
test written this way. You can see it at:
`test/linux-generic/validation/api/shmem`

==== Marking validation tests as inactive

The general policy is that a full run of the validation suite (a `make check`)
must pass at all times. However a particular platform may have one or more test
cases that are known to be unimplemented either during development or
permanently, so to avoid these test cases being reported as failures it's useful
to be able to skip them. This can be achieved by creating a new test executable
(still on the platform side), giving the platform specific initialization code
the opportunity to modify the registered tests in order to mark unwanted tests
as inactive while leaving the remaining tests active. It's important that the
unwanted tests are still registered with the test framework to allow the fact
that they're not being tested to be recorded.

The `odp_cunit_update()` function is intended for this purpose, it is used to
modify the properties of previously registered tests, for example to mark them
as inactive. Inactive tests are registered with the test framework but aren't
executed and will be recorded as inactive in test reports.

In `test/common_plat/validation/api/foo/foo.c`, define all
validation tests for the 'foo' module:

[source,c]
------------------
odp_testinfo_t foo_tests[] = {
	ODP_TEST_INFO(foo_test_a),
	ODP_TEST_INFO(foo_test_b),
	ODP_TEST_INFO_NULL
};

odp_suiteinfo_t foo_suites[] = {
	{"Foo", foo_suite_init, foo_suite_term, foo_tests},
	ODP_SUITE_INFO_NULL
};
------------------

In `<platform>/validation/api/foo/foo_main.c`, register all the tests defined in
the `foo` module, then mark a single specific test case as inactive:

[source,c]
------------------
static odp_testinfo_t foo_tests_updates[] = {
	ODP_TEST_INFO_INACTIVE(foo_test_b),
	ODP_TEST_INFO_NULL
};

static odp_suiteinfo_t foo_suites_updates[] = {
	{"Foo", foo_suite_init, foo_suite_term, foo_tests_updates},
	ODP_SUITE_INFO_NULL
};

int foo_main(void)
{
	int ret = odp_cunit_register(foo_suites);

	if (ret == 0)
		ret = odp_cunit_update(foo_suites_updates);

	if (ret == 0)
		ret = odp_cunit_run();

	return ret;
}
------------------

So `foo_test_a` will be executed and `foo_test_b` is inactive.

It's expected that early in the development cycle of a new implementation the
inactive list will be quite long, but it should shrink over time as more parts
of the API are implemented.

==== Conditional Tests

Some tests may require specific conditions to make sense: for instance, on
pktio, checking that sending a packet larger than the MTU is rejected only makes
sense if packets can indeed, on that ODP implementation, exceed the MTU.
A test can be marked conditional as follows:

[source,c]
------------------
odp_testinfo_t foo_tests[] = {
	...
	ODP_TEST_INFO_CONDITIONAL(foo_test_x, foo_check_x),
	...
	ODP_TEST_INFO_NULL
};

odp_suiteinfo_t foo_suites[] = {
	{"Foo", foo_suite_init, foo_suite_term, foo_tests},
	ODP_SUITE_INFO_NULL
};
------------------

Foo_test_x is the usual test function. Foo_check_x is the test precondition,
i.e. a function returning a Boolean (int).
It is called before the test suite is started. If it returns true, the
test (foo_test_x) is run. If the precondition function (foo_check_x above)
returns false, the test is not relevant (or impossible to perform) and it will
be skipped.

=================
*Note*

Conditional tests can be marked as inactive, keeping the precondition
function. Both the test and the precondition function will be skipped,
but re-activating the test is then just a matter of changing back the macro
from ODP_TEST_INFO_INACTIVE to ODP_TEST_INFO_CONDITIONAL:

[source,c]
------------------
	...
	/* active conditional test */
	ODP_TEST_INFO_CONDITIONAL(foo_test_x, foo_check_x),

	/* inactive conditional test */
	ODP_TEST_INFO_INACTIVE(foo_test_y, foo_check_y),
	...
------------------
=================

==== helper usage ====

The tests (both platform agnostic and platform dependent tests) make use of
a set of functions defined in a helper library. The helper library tries to
abstract and regroup common actions that applications may perform but
which are not part of the ODP API (i.e. mostly OS system calls).
Using these functions is recommended, as running the tests on a different OS
could (hopefully) be as simple as changing the OS related helper lib.

In the linux helper, two functions are given to create and join ODP threads:

`odph_odpthreads_create()`

`odph_odpthreads_join()`

These two functions abstract what an ODP thread really is and their usage
is recommended as they would be implemented in other OS`s helper lib.

Five older functions exist to tie and ODP thread to a specific implementation:

`odph_linux_pthread_create()`

`odph_linux_pthread_join()`

`odph_linux_process_fork_n()`

`odph_linux_process_fork()`

`odph_linux_process_wait_n()`

The usage of these functions should not occur within ODP examples nor tests.
The usage of these functions in other application is not recommended.

[[typedefs]]
== ODP Abstract Types and Implementation Typedefs
ODP APIs are defined to be abstract and operate on abstract types. For example,
ODP APIs that perform packet manipulation manipulate objects of type
`odp_packet_t`. Queues are represented by objects of type `odp_queue_t`, etc.

Since the C compiler cannot compile code that has unresolved abstract types,
the first task of each ODP implementation is to decide how it wishes to
represent each of these abstract types and to supply appropriate `typedef`
definitions for them to make ODP applications compilable on this platform.

It is recommended that however a platform wishes to represent ODP abstract
types, that it do so in a strongly typed manner. Using strong types means
that an application that tries to pass a variable of type `odp_packet_t` to
an API that expects an argument of type `odp_queue_t`, for example, will result
in a compilation error rather than some difficult to debug runtime failure.

The *odp-linux* reference implementation defines all ODP abstract types strongly
using a set of utility macros contained in
`platform/linux-generic/include/odp/api/plat/strong_types.h`. These macros
can be used or modified as desired by other implementations to achieve strong
typing of their typedefs.

=== Typedef approaches
ODP abstract types serve two distinct purposes that each implementation must
consider. First, they shield applications from implementation internals, thus
facilitating ODP application portability. Equally important, however, is that
implementations choose typdefs and representations that permit the
implementation to realize ODP APIs efficiently. This typically means that the
handles defined by typedefs are either a pointer to an implementation-defined
struct or else an index into an implementation-defined resource table. The two
LNG-provided ODP reference implementations illustrate both of these approaches.
The *odp-dpdk* implementation follows the former approach (pointers) as this
offers the highest performance. For example, in *odp-dpdk* an
`odp_packet_t` is a pointer to an `rte_mbuf` struct, which is how DPDK
represents packets. The *odp-linux* implementation, by contrast, uses indices
as this permits more robust validation support while still being highly
efficient. In general, software-based implementations will typically favor
pointers while hardware-based implementations will typically favor indices.

=== ABI Considerations
An _Application Binary Interface_ is a specification of the _representation_
of an API that guarantees that applications can move between implementations
of an API without recompilation. ABIs thus go beyond the basic source-code
portability guarantees provided by APIs to permit binary portability as well.

It is important to note that ODP neither defines nor prohibits the specification
of ABIs. This is because ODP itself is an _Abstract API Specification_. As
noted earlier, abstract APIs cannot be compiled in the absence of completion
by an implementation that instantiates them, so the question of ABIs is
really a question of representation agreement between multiple ODP
implementations. If two or more ODP implementations agree on things like
typedefs, endianness, alignments, etc., then they are defining an ABI which
would permit ODP applications compiled to that common set of instantiations
to inter operate at a binary as well as source level.

==== Traditional ABI
ABIs can be defined at two levels. The simplest ABI is within a specific
Instruction Set Architecture (ISA). So, for example, an ABI might be defined
among ODP implementations for the AArch64 or x86 architectures. This
traditional approach is shown here:

.Traditional ABI Structure
image::abi_traditional.svg[align="center"]

In the traditional approach, multiple target platforms agree on a common set
of typedefs, etc. so that the resulting output from compilation is directly
executable on any platform that subscribes to that ABI. Adding a new platform
in this approach simply requires that platform to accept the existing ABI
specification. Note that since the output of compilation in a traditional ABI
is a ISA-specific binary that applications cannot offer binary compatibility
across platforms that use different ISAs.

==== Bitcode based ABI
An ABI an also be defined at a higher level by moving to a more sophisticated
tool chain (such as is possible using LLVM) that implements a split
compilation model. In this model, the output from a compiler is not directly
executable. Rather it is a standardized intermediate representation called
_bitcode_ that must be further processed to result in an executable image as
shown here:

.Bitcode ABI Structure
image::abi_llvm.svg[align="center"]

The key to this model is that the platform linking and optimization that is
needed to create a platform executable is a system rather than a developer
responsibility. The developer's output is a universal bitcode binary. From
here, the library system creates a series of _managed binaries_ that result
from performing link-time optimization against a set of platform definitions.
When a universal application is to run on a specific target platform, the
library system selects the appropriate managed binary for that target platform
and loads and runs it.

Adding a new platform in this approach involves adding the definition for that
platform to the library system so that a managed binary for it can be created
and deployed as needed. This occurs without developer involvement since the
bitcode format that is input to this backend process is independent of the
specific target platform. Note also that since bitcode is not tied to any ISA,
applications using bitcode ABIs are binary portable between platforms that use
different ISAs. This occurs without loss of efficiency because the process of
creating a managed binary is itself a secondary compilation and optimization
step. The difference is that performing this step is a system rather than a
developer responsibility.

include::../glossary.adoc[]