diff options
Diffstat (limited to 'docs/devel/build-system.txt')
| -rw-r--r-- | docs/devel/build-system.txt | 525 |
1 files changed, 0 insertions, 525 deletions
diff --git a/docs/devel/build-system.txt b/docs/devel/build-system.txt deleted file mode 100644 index 52501f2ad9..0000000000 --- a/docs/devel/build-system.txt +++ /dev/null @@ -1,525 +0,0 @@ - The QEMU build system architecture - ================================== - -This document aims to help developers understand the architecture of the -QEMU build system. As with projects using GNU autotools, the QEMU build -system has two stages, first the developer runs the "configure" script -to determine the local build environment characteristics, then they run -"make" to build the project. There is about where the similarities with -GNU autotools end, so try to forget what you know about them. - - -Stage 1: configure -================== - -The QEMU configure script is written directly in shell, and should be -compatible with any POSIX shell, hence it uses #!/bin/sh. An important -implication of this is that it is important to avoid using bash-isms on -development platforms where bash is the primary host. - -In contrast to autoconf scripts, QEMU's configure is expected to be -silent while it is checking for features. It will only display output -when an error occurs, or to show the final feature enablement summary -on completion. - -Adding new checks to the configure script usually comprises the -following tasks: - - - Initialize one or more variables with the default feature state. - - Ideally features should auto-detect whether they are present, - so try to avoid hardcoding the initial state to either enabled - or disabled, as that forces the user to pass a --enable-XXX - / --disable-XXX flag on every invocation of configure. - - - Add support to the command line arg parser to handle any new - --enable-XXX / --disable-XXX flags required by the feature XXX. - - - Add information to the help output message to report on the new - feature flag. - - - Add code to perform the actual feature check. As noted above, try to - be fully dynamic in checking enablement/disablement. - - - Add code to print out the feature status in the configure summary - upon completion. - - - Add any new makefile variables to $config_host_mak on completion. - - -Taking (a simplified version of) the probe for gnutls from configure, -we have the following pieces: - - # Initial variable state - gnutls="" - - ..snip.. - - # Configure flag processing - --disable-gnutls) gnutls="no" - ;; - --enable-gnutls) gnutls="yes" - ;; - - ..snip.. - - # Help output feature message - gnutls GNUTLS cryptography support - - ..snip.. - - # Test for gnutls - if test "$gnutls" != "no"; then - if ! $pkg_config --exists "gnutls"; then - gnutls_cflags=`$pkg_config --cflags gnutls` - gnutls_libs=`$pkg_config --libs gnutls` - libs_softmmu="$gnutls_libs $libs_softmmu" - libs_tools="$gnutls_libs $libs_tools" - QEMU_CFLAGS="$QEMU_CFLAGS $gnutls_cflags" - gnutls="yes" - elif test "$gnutls" = "yes"; then - feature_not_found "gnutls" "Install gnutls devel" - else - gnutls="no" - fi - fi - - ..snip.. - - # Completion feature summary - echo "GNUTLS support $gnutls" - - ..snip.. - - # Define make variables - if test "$gnutls" = "yes" ; then - echo "CONFIG_GNUTLS=y" >> $config_host_mak - fi - - -Helper functions ----------------- - -The configure script provides a variety of helper functions to assist -developers in checking for system features: - - - do_cc $ARGS... - - Attempt to run the system C compiler passing it $ARGS... - - - do_cxx $ARGS... - - Attempt to run the system C++ compiler passing it $ARGS... - - - compile_object $CFLAGS - - Attempt to compile a test program with the system C compiler using - $CFLAGS. The test program must have been previously written to a file - called $TMPC. - - - compile_prog $CFLAGS $LDFLAGS - - Attempt to compile a test program with the system C compiler using - $CFLAGS and link it with the system linker using $LDFLAGS. The test - program must have been previously written to a file called $TMPC. - - - has $COMMAND - - Determine if $COMMAND exists in the current environment, either as a - shell builtin, or executable binary, returning 0 on success. - - - path_of $COMMAND - - Return the fully qualified path of $COMMAND, printing it to stdout, - and returning 0 on success. - - - check_define $NAME - - Determine if the macro $NAME is defined by the system C compiler - - - check_include $NAME - - Determine if the include $NAME file is available to the system C - compiler - - - write_c_skeleton - - Write a minimal C program main() function to the temporary file - indicated by $TMPC - - - feature_not_found $NAME $REMEDY - - Print a message to stderr that the feature $NAME was not available - on the system, suggesting the user try $REMEDY to address the - problem. - - - error_exit $MESSAGE $MORE... - - Print $MESSAGE to stderr, followed by $MORE... and then exit from the - configure script with non-zero status - - - query_pkg_config $ARGS... - - Run pkg-config passing it $ARGS. If QEMU is doing a static build, - then --static will be automatically added to $ARGS - - -Stage 2: makefiles -================== - -The use of GNU make is required with the QEMU build system. - -Although the source code is spread across multiple subdirectories, the -build system should be considered largely non-recursive in nature, in -contrast to common practices seen with automake. There is some recursive -invocation of make, but this is related to the things being built, -rather than the source directory structure. - -QEMU currently supports both VPATH and non-VPATH builds, so there are -three general ways to invoke configure & perform a build. - - - VPATH, build artifacts outside of QEMU source tree entirely - - cd ../ - mkdir build - cd build - ../qemu/configure - make - - - VPATH, build artifacts in a subdir of QEMU source tree - - mkdir build - cd build - ../configure - make - - - non-VPATH, build artifacts everywhere - - ./configure - make - -The QEMU maintainers generally recommend that a VPATH build is used by -developers. Patches to QEMU are expected to ensure VPATH build still -works. - - -Module structure ----------------- - -There are a number of key outputs of the QEMU build system: - - - Tools - qemu-img, qemu-nbd, qga (guest agent), etc - - System emulators - qemu-system-$ARCH - - Userspace emulators - qemu-$ARCH - - Unit tests - -The source code is highly modularized, split across many files to -facilitate building of all of these components with as little duplicated -compilation as possible. There can be considered to be two distinct -groups of files, those which are independent of the QEMU emulation -target and those which are dependent on the QEMU emulation target. - -In the target-independent set lives various general purpose helper code, -such as error handling infrastructure, standard data structures, -platform portability wrapper functions, etc. This code can be compiled -once only and the .o files linked into all output binaries. - -In the target-dependent set lives CPU emulation, device emulation and -much glue code. This sometimes also has to be compiled multiple times, -once for each target being built. - -The utility code that is used by all binaries is built into a -static archive called libqemuutil.a, which is then linked to all the -binaries. In order to provide hooks that are only needed by some of the -binaries, code in libqemuutil.a may depend on other functions that are -not fully implemented by all QEMU binaries. Dummy stubs for all these -functions are also provided by this library, and will only be linked -into the binary if the real implementation is not present. In a way, -the stubs can be thought of as a portable implementation of the weak -symbols concept. - -All binaries should link to libqemuutil.a, e.g.: - - qemu-img$(EXESUF): qemu-img.o ..snip.. libqemuutil.a - - -Windows platform portability ----------------------------- - -On Windows, all binaries have the suffix '.exe', so all Makefile rules -which create binaries must include the $(EXESUF) variable on the binary -name. e.g. - - qemu-img$(EXESUF): qemu-img.o ..snip.. - -This expands to '.exe' on Windows, or '' on other platforms. - -A further complication for the system emulator binaries is that -two separate binaries need to be generated. - -The main binary (e.g. qemu-system-x86_64.exe) is linked against the -Windows console runtime subsystem. These are expected to be run from a -command prompt window, and so will print stderr to the console that -launched them. - -The second binary generated has a 'w' on the end of its name (e.g. -qemu-system-x86_64w.exe) and is linked against the Windows graphical -runtime subsystem. These are expected to be run directly from the -desktop and will open up a dedicated console window for stderr output. - -The Makefile.target will generate the binary for the graphical subsystem -first, and then use objcopy to relink it against the console subsystem -to generate the second binary. - - -Object variable naming ----------------------- - -The QEMU convention is to define variables to list different groups of -object files. These are named with the convention $PREFIX-obj-y. For -example the libqemuutil.a file will be linked with all objects listed -in a variable 'util-obj-y'. So, for example, util/Makefile.obj will -contain a set of definitions looking like - - util-obj-y += bitmap.o bitops.o hbitmap.o - util-obj-y += fifo8.o - util-obj-y += acl.o - util-obj-y += error.o qemu-error.o - -When there is an object file which needs to be conditionally built based -on some characteristic of the host system, the configure script will -define a variable for the conditional. For example, on Windows it will -define $(CONFIG_POSIX) with a value of 'n' and $(CONFIG_WIN32) with a -value of 'y'. It is now possible to use the config variables when -listing object files. For example, - - util-obj-$(CONFIG_WIN32) += oslib-win32.o qemu-thread-win32.o - util-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o - -On Windows this expands to - - util-obj-y += oslib-win32.o qemu-thread-win32.o - util-obj-n += oslib-posix.o qemu-thread-posix.o - -Since libqemutil.a links in $(util-obj-y), the POSIX specific files -listed against $(util-obj-n) are ignored on the Windows platform builds. - - -CFLAGS / LDFLAGS / LIBS handling --------------------------------- - -There are many different binaries being built with differing purposes, -and some of them might even be 3rd party libraries pulled in via git -submodules. As such the use of the global CFLAGS variable is generally -avoided in QEMU, since it would apply to too many build targets. - -Flags that are needed by any QEMU code (i.e. everything *except* GIT -submodule projects) are put in $(QEMU_CFLAGS) variable. For linker -flags the $(LIBS) variable is sometimes used, but a couple of more -targeted variables are preferred. $(libs_softmmu) is used for -libraries that must be linked to system emulator targets, $(LIBS_TOOLS) -is used for tools like qemu-img, qemu-nbd, etc and $(LIBS_QGA) is used -for the QEMU guest agent. There is currently no specific variable for -the userspace emulator targets as the global $(LIBS), or more targeted -variables shown below, are sufficient. - -In addition to these variables, it is possible to provide cflags and -libs against individual source code files, by defining variables of the -form $FILENAME-cflags and $FILENAME-libs. For example, the curl block -driver needs to link to the libcurl library, so block/Makefile defines -some variables: - - curl.o-cflags := $(CURL_CFLAGS) - curl.o-libs := $(CURL_LIBS) - -The scope is a little different between the two variables. The libs get -used when linking any target binary that includes the curl.o object -file, while the cflags get used when compiling the curl.c file only. - - -Statically defined files ------------------------- - -The following key files are statically defined in the source tree, with -the rules needed to build QEMU. Their behaviour is influenced by a -number of dynamically created files listed later. - -- Makefile - -The main entry point used when invoking make to build all the components -of QEMU. The default 'all' target will naturally result in the build of -every component. The various tools and helper binaries are built -directly via a non-recursive set of rules. - -Each system/userspace emulation target needs to have a slightly -different set of make rules / variables. Thus, make will be recursively -invoked for each of the emulation targets. - -The recursive invocation will end up processing the toplevel -Makefile.target file (more on that later). - - -- */Makefile.objs - -Since the source code is spread across multiple directories, the rules -for each file are similarly modularized. Thus each subdirectory -containing .c files will usually also contain a Makefile.objs file. -These files are not directly invoked by a recursive make, but instead -they are imported by the top level Makefile and/or Makefile.target - -Each Makefile.objs usually just declares a set of variables listing the -.o files that need building from the source files in the directory. They -will also define any custom linker or compiler flags. For example in -block/Makefile.objs - - block-obj-$(CONFIG_LIBISCSI) += iscsi.o - block-obj-$(CONFIG_CURL) += curl.o - - ..snip... - - iscsi.o-cflags := $(LIBISCSI_CFLAGS) - iscsi.o-libs := $(LIBISCSI_LIBS) - curl.o-cflags := $(CURL_CFLAGS) - curl.o-libs := $(CURL_LIBS) - -If there are any rules defined in the Makefile.objs file, they should -all use $(obj) as a prefix to the target, e.g. - - $(obj)/generated-tcg-tracers.h: $(obj)/generated-tcg-tracers.h-timestamp - - -- Makefile.target - -This file provides the entry point used to build each individual system -or userspace emulator target. Each enabled target has its own -subdirectory. For example if configure is run with the argument -'--target-list=x86_64-softmmu', then a sub-directory 'x86_64-softmu' -will be created, containing a 'Makefile' which symlinks back to -Makefile.target - -So when the recursive '$(MAKE) -C x86_64-softmmu' is invoked, it ends up -using Makefile.target for the build rules. - - -- rules.mak - -This file provides the generic helper rules for invoking build tools, in -particular the compiler and linker. This also contains the magic (hairy) -'unnest-vars' function which is used to merge the variable definitions -from all Makefile.objs in the source tree down into the main Makefile -context. - - -- default-configs/*.mak - -The files under default-configs/ control what emulated hardware is built -into each QEMU system and userspace emulator targets. They merely -contain a long list of config variable definitions. For example, -default-configs/x86_64-softmmu.mak has: - - include pci.mak - include sound.mak - include usb.mak - CONFIG_QXL=$(CONFIG_SPICE) - CONFIG_VGA_ISA=y - CONFIG_VGA_CIRRUS=y - CONFIG_VMWARE_VGA=y - CONFIG_VIRTIO_VGA=y - ...snip... - -These files rarely need changing unless new devices / hardware need to -be enabled for a particular system/userspace emulation target - - -- tests/Makefile - -Rules for building the unit tests. This file is included directly by the -top level Makefile, so anything defined in this file will influence the -entire build system. Care needs to be taken when writing rules for tests -to ensure they only apply to the unit test execution / build. - -- tests/docker/Makefile.include - -Rules for Docker tests. Like tests/Makefile, this file is included -directly by the top level Makefile, anything defined in this file will -influence the entire build system. - -- po/Makefile - -Rules for building and installing the binary message catalogs from the -text .po file sources. This almost never needs changing for any reason. - - -Dynamically created files -------------------------- - -The following files are generated dynamically by configure in order to -control the behaviour of the statically defined makefiles. This avoids -the need for QEMU makefiles to go through any pre-processing as seen -with autotools, where Makefile.am generates Makefile.in which generates -Makefile. - - -- config-host.mak - -When configure has determined the characteristics of the build host it -will write a long list of variables to config-host.mak file. This -provides the various install directories, compiler / linker flags and a -variety of CONFIG_* variables related to optionally enabled features. -This is imported by the top level Makefile in order to tailor the build -output. - -The variables defined here are those which are applicable to all QEMU -build outputs. Variables which are potentially different for each -emulator target are defined by the next file... - -It is also used as a dependency checking mechanism. If make sees that -the modification timestamp on configure is newer than that on -config-host.mak, then configure will be re-run. - - -- config-host.h - -The config-host.h file is used by source code to determine what features -are enabled. It is generated from the contents of config-host.mak using -the scripts/create_config program. This extracts all the CONFIG_* variables, -most of the HOST_* variables and a few other misc variables from -config-host.mak, formatting them as C preprocessor macros. - - -- $TARGET-NAME/config-target.mak - -TARGET-NAME is the name of a system or userspace emulator, for example, -x86_64-softmmu denotes the system emulator for the x86_64 architecture. -This file contains the variables which need to vary on a per-target -basis. For example, it will indicate whether KVM or Xen are enabled for -the target and any other potential custom libraries needed for linking -the target. - - -- $TARGET-NAME/config-devices.mak - -TARGET-NAME is again the name of a system or userspace emulator. The -config-devices.mak file is automatically generated by make using the -scripts/make_device_config.sh program, feeding it the -default-configs/$TARGET-NAME file as input. - - -- $TARGET-NAME/Makefile - -This is the entrypoint used when make recurses to build a single system -or userspace emulator target. It is merely a symlink back to the -Makefile.target in the top level. - - -Useful make targets -=================== - -- help - - Print a help message for the most common build targets. - -- print-VAR - - Print the value of the variable VAR. Useful for debugging the build - system. |