aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-02-27docs/system: convert Texinfo documentation to rSTpaolo-docsPaolo Bonzini
Apart from targets.rst, which was written by hand, this is an automated conversion obtained with the following command: makeinfo --force -o - --docbook \ -D 'qemu_system_x86 QEMU_SYSTEM_X86_MACRO' \ -D 'qemu_system QEMU_SYSTEM_MACRO' \ $texi | pandoc -f docbook -t rst+smart | perl -e ' $/=undef; $_ = <>; s/^- − /- /gm; s/QEMU_SYSTEM_MACRO/|qemu_system|/g; s/QEMU_SYSTEM_X86_MACRO/|qemu_system_x86|/g; s/(?=::\n\n +\|qemu)/.. parsed-literal/g; s/:\n\n::$/::/gm; print' > $rst In addition, the following changes were made manually: - target-i386.rst and target-mips.rst: replace CPU model documentation with an include directive - monitor.rst: replace the command section with a comment - images.rst: add toctree - target-arm.rst: Replace use of :math: (which Sphinx complains about) with :sup:, and hide it behind |I2C| and |I2C| substitutions. Content that is not @included remains exclusive to qemu-doc.texi. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-19-pbonzini@redhat.com [PMM: Fixed target-arm.rst use of :math:; remove out of date note about images.rst from commit message; fixed expansion of |qemu_system_x86|; use parsed-literal in invocation.rst when we want to use |qemu_system_x86|; fix incorrect subsection level for "OS requirements" in target-i386.rst; fix incorrect syntax for making links to other sections of the manual] Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27docs/system: convert the documentation of deprecated features to rST.Peter Maydell
We put the whole of this document into the system manual, though technically a few parts of it apply to qemu-img or qemu-nbd which are otherwise documented in tools/. We only make formatting fixes, except for one use of 'appendix' which we change to 'section' because this isn't an appendix in the Sphinx manual. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-18-pbonzini@redhat.com
2020-02-27docs/system: convert managed startup to rST.Peter Maydell
Fix one typo in the process and format more option and command names as literal text, but make no significant changes to the content. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-17-pbonzini@redhat.com
2020-02-27docs/system: Convert security.texi to rST formatPeter Maydell
security.texi is included from qemu-doc.texi but is not used in the qemu.1 manpage. So we can do a straightforward conversion of the contents, which go into the system manual. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-16-pbonzini@redhat.com
2020-02-27docs/system: Convert qemu-cpu-models.texi to rSTKashyap Chamarthy
This doc was originally written by Daniel P. Berrangé <berrange@redhat.com>, introduced via commit[1]: 2544e9e4aa (docs: add guidance on configuring CPU models for x86, 2018-06-27). In this patch: - 1-1 conversion of Texinfo to rST, besides a couple of minor tweaks that are too trivial to mention. (Thanks to Stephen Finucane on IRC for the suggestion to use rST "definition lists" instead of bullets in some places.) Further modifications will be done via a separate patch. - rST and related infra changes: manual page generation, Makefile fixes, clean up references to qemu-cpu-models.texi, update year in the copyright notice, etc. [1] https://git.qemu.org/?p=qemu.git;a=commit;h=2544e9e4aa As part of the conversion, we use a more generic 'author' attribution for the manpage than we previously had, as agreed with the original author Dan Berrange. Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-15-pbonzini@redhat.com [Move macros to defs.rst.inc, split in x86 and MIPS parts, make qemu-cpu-models.rst a standalone document. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> [PMM: Move defs.rst.inc setup to its own commit; fix minor issues with MAINTAINERS file updates; drop copyright date change; keep capitalization of "QEMU Project developers" consistent with other uses; minor Makefile fixups] Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27docs: Create defs.rst.inc as a place to define substitutionsPeter Maydell
Rather than accumulating generally useful rST substitution definitions in individual rST files, create a defs.rst.inc where we can define these. To start with it has the |qemu_system| definition from qemu-block-drivers.rst. Since we're making |qemu_system| generally available, add the ``...`` literal markup so it can be used in running text as well as inside a parsed-literal block. Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27docs/system: put qemu-block-drivers body in an included filePaolo Bonzini
This removes the "only" directives, and lets us use the conventional "DESCRIPTION" section in the manpage. This temporarily drops the qemu-block-drivers documentation from the system manual, but it will be put back (in the right place in the toctree) in a later commit. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-14-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> [PMM: Added commit message note about temporarily losing qemu-block-drivers from the system manual] Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: remove indices other than findexPaolo Bonzini
These indices are not well-maintained, and pandoc also chokes on the directives. Just nuke them. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-13-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: move included files to docs/systemPaolo Bonzini
Since qemu-doc.texi is mostly including files from docs/system, move the existing include files there for consistency. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-12-pbonzini@redhat.com [PMM: update MAINTAINERS line for qemu-option-trace.texi] Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: move qemu-tech.texi into main sectionPaolo Bonzini
The only remaining content in qemu-tech.texi is a few paragraphs about managed start up options. Move them in the main section about full system emulation. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-11-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: Remove the "CPU emulation" part of the "Implementation notes"Peter Maydell
The "CPU emulation" part of the "Implementation notes" in qemu-tech.texi looks like it is documenting what features of various CPUs we do or don't emulate. However: * it covers only six of our 21 guest architectures * the last time anybody updated it for actual content was in 2011/2012 for Xtensa; the content for the other five architectures is even older, being from 2008 or before! What we have is out of date, misleading and incomplete. Just delete this part of the document rather than trying to convert it to rST. (It would be nice eventually to have documentation of the scope and limitations of our emulation; but we will want to separate out the generic "system emulation" information from the parts that are specific to linux-user anyway, as they will be in different manuals.) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200226113034.6741-10-pbonzini@redhat.com Message-Id: <20200225154121.21116-3-peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-27qemu-doc: split target sections to separate filesPaolo Bonzini
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-9-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: move system requirements chapter inside PC sectionPaolo Bonzini
The system requirements documented in this chapter are limited to x86 KVM targets. Clean them up and move them to the target section. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-8-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: extract common system emulator documentation from the PC sectionPaolo Bonzini
Move the section on PC peripherals together with other targets. While some x86-specific information remains in the main system emulation chapter, it can be tackled more easily a section at a time. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-7-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: split qemu-doc.texi in multiple filesPaolo Bonzini
In order to facilitate the reorganization of qemu-doc.texi content, as well as the conversion to rST/Sphinx, split it in multiple .texi files that are included from docs/system. The "other devices" section is renamed to ivshmem and placed last. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-6-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: split CPU models doc between MIPS and x86 partsPaolo Bonzini
The MIPS CPU models end up in the middle of the PC documentation. Move them to a separate file so that they can be placed in the right section. The man page still includes both x86 and MIPS content. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-5-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27texi2pod: parse @include directives outside "@c man" blocksPaolo Bonzini
This enables splitting the huge qemu-doc.texi file and keeping parallel Texinfo and rST versions of the documentation. texi2pod is not going to live much longer and hardly anyone cares about its upstream status, so the temporary fork should be acceptable. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-4-pbonzini@redhat.com Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: remove target OS documentationPaolo Bonzini
This section covers OSes up to Windows 2000, and as such it is mostly obsolete. Zap it. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-3-pbonzini@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-27qemu-doc: convert user-mode emulation to a separate Sphinx manualPaolo Bonzini
The final addition to the set of QEMU manuals is the user-mode emulation manual, which right now is included in qemu-doc.texi. Extract it and convert it to rST, so that qemu-doc.texi covers only full system emulation. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200226113034.6741-2-pbonzini@redhat.com [PMM: Fix makefile conflicts; add user manual to index.rst and index.html.in; don't specify empty man_pages list; fixed a few comments to say 'user' rather than 'system'] Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-25Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* device_del fix (Julia) * FXAM fix (myself) * memdev refactoring (Igor) * memory region API cleanups (Peter, Philippe) * ioeventfd optimization (Stefan) * new WHPX maintainer (Sunil) * Large guest startup optimizations (Chen) # gpg: Signature made Tue 25 Feb 2020 12:42:24 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full] # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full] # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream: (104 commits) WHPX: Assigning maintainer for Windows Hypervisor Platform accel/kvm: Check ioctl(KVM_SET_USER_MEMORY_REGION) return value target/i386: check for empty register in FXAM qdev-monitor: Forbid repeated device_del mem-prealloc: optimize large guest startup memory: batch allocate ioeventfds[] in address_space_update_ioeventfds() Avoid cpu_physical_memory_rw() with a constant is_write argument Let cpu_[physical]_memory() calls pass a boolean 'is_write' argument exec: Let cpu_[physical]_memory API use a boolean 'is_write' argument Avoid address_space_rw() with a constant is_write argument Let address_space_rw() calls pass a boolean 'is_write' argument exec: Let address_space_unmap() use a boolean 'is_write' argument hw/virtio: Let vhost_memory_map() use a boolean 'is_write' argument hw/virtio: Let virtqueue_map_iovec() use a boolean 'is_write' argument hw/ide: Let the DMAIntFunc prototype use a boolean 'is_write' argument hw/ide/internal: Remove unused DMARestartFunc typedef Remove unnecessary cast when using the cpu_[physical]_memory API exec: Let the cpu_[physical]_memory API use void pointer arguments Remove unnecessary cast when using the address_space API hw/net: Avoid casting non-const pointer, use address_space_write() ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-25Merge branch 'exec_rw_const_v4' of https://github.com/philmd/qemu into HEADPaolo Bonzini
2020-02-25Merge remote-tracking branch 'remotes/pmaydell/tags/pull-docs-20200225' into ↵Peter Maydell
staging * create a new 'tools' manual, and move the documentation for qemu-img, qemu-nbd, virtfs-proxy-helper, qemu-trace-stap and virtiofsd to it # gpg: Signature made Tue 25 Feb 2020 11:00:16 GMT # gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE # gpg: issuer "peter.maydell@linaro.org" # gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate] # gpg: aka "Peter Maydell <pmaydell@gmail.com>" [ultimate] # gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate] # Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE * remotes/pmaydell/tags/pull-docs-20200225: docs: Move tools documentation to tools manual docs: Move qemu-option-trace.rst.inc to docs/ docs: Create new 'tools' manual Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-25docs: Move tools documentation to tools manualpull-docs-20200225Peter Maydell
Move the following tools documentation files to the new tools manual: docs/interop/qemu-img.rst docs/interop/qemu-nbd.rst docs/interop/virtfs-proxy-helper.rst docs/interop/qemu-trace-stap.rst docs/interop/virtiofsd.rst Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200217155415.30949-4-peter.maydell@linaro.org
2020-02-25docs: Move qemu-option-trace.rst.inc to docs/Peter Maydell
The qemu-option-trace.rst.inc file contains a rST documentation fragment which describes trace options common to qemu-nbd and qemu-img. We put this file into interop/, but we'd like to move the qemu-nbd and qemu-img files into the tools/ manual. We could move the .rst.inc file along with them, but we're eventually going to want to use it for the main QEMU binary options documentation too, and that will be in system/. So move qemu-option-trace.rst.inc to the top-level docs/ directory, where all these files can include it via .. include:: ../qemu-option-trace.rst.inc This does have the slight downside that we now need to explicitly tell Make which manuals use this file rather than relying on a wildcard for all .rst.inc in the manual. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200217155415.30949-3-peter.maydell@linaro.org
2020-02-25docs: Create new 'tools' manualPeter Maydell
Some of the documentation for QEMU "tools" which are standalone binaries like qemu-img is an awkward fit in our current 5-manual split. We've put it into "interop", but they're not really about interoperability. Create a new top level manual "tools" which will be a better home for this documentation. This commit creates an empty initial manual; we will move the relevant documentation files in a subsequent commit. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200217155415.30949-2-peter.maydell@linaro.org
2020-02-25Merge tag 'patchew/20200219160953.13771-1-imammedo@redhat.com' of ↵Paolo Bonzini
https://github.com/patchew-project/qemu into HEAD This series removes ad hoc RAM allocation API (memory_region_allocate_system_memory) and consolidates it around hostmem backend. It allows to * resolve conflicts between global -mem-prealloc and hostmem's "policy" option, fixing premature allocation before binding policy is applied * simplify complicated memory allocation routines which had to deal with 2 ways to allocate RAM. * reuse hostmem backends of a choice for main RAM without adding extra CLI options to duplicate hostmem features. A recent case was -mem-shared, to enable vhost-user on targets that don't support hostmem backends [1] (ex: s390) * move RAM allocation from individual boards into generic machine code and provide them with prepared MemoryRegion. * clean up deprecated NUMA features which were tied to the old API (see patches) - "numa: remove deprecated -mem-path fallback to anonymous RAM" - (POSTPONED, waiting on libvirt side) "forbid '-numa node,mem' for 5.0 and newer machine types" - (POSTPONED) "numa: remove deprecated implicit RAM distribution between nodes" Introduce a new machine.memory-backend property and wrapper code that aliases global -mem-path and -mem-alloc into automatically created hostmem backend properties (provided memory-backend was not set explicitly given by user). A bulk of trivial patches then follow to incrementally convert individual boards to using machine.memory-backend provided MemoryRegion. Board conversion typically involves: * providing MachineClass::default_ram_size and MachineClass::default_ram_id so generic code could create default backend if user didn't explicitly provide memory-backend or -m options * dropping memory_region_allocate_system_memory() call * using convenience MachineState::ram MemoryRegion, which points to MemoryRegion allocated by ram-memdev On top of that for some boards: * missing ram_size checks are added (typically it were boards with fixed ram size) * ram_size fixups are replaced by checks and hard errors, forcing user to provide correct "-m" values instead of ignoring it and continuing running. After all boards are converted, the old API is removed and memory allocation routines are cleaned up.
2020-02-25WHPX: Assigning maintainer for Windows Hypervisor PlatformSunil Muthuswamy
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com> Message-Id: <SN4PR2101MB0880E245954826FD91C9D67DC0110@SN4PR2101MB0880.namprd21.prod.outlook.com> Reviewed-by: Justin Terry (VM) <juterry@microsoft.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-25accel/kvm: Check ioctl(KVM_SET_USER_MEMORY_REGION) return valuePhilippe Mathieu-Daudé
kvm_vm_ioctl() can fail, check its return value, and log an error when it failed. This fixes Coverity CID 1412229: Unchecked return value (CHECKED_RETURN) check_return: Calling kvm_vm_ioctl without checking return value Reported-by: Coverity (CID 1412229) Fixes: 235e8982ad3 ("support using KVM_MEM_READONLY flag for regions") Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Message-Id: <20200221163336.2362-1-philmd@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-25target/i386: check for empty register in FXAMPaolo Bonzini
The fxam instruction returns the wrong result after fdecstp or after an underflow. Check fptags to handle this. Reported-by: <chengang@emindsoft.com.cn> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-25qdev-monitor: Forbid repeated device_delJulia Suvorova
Device unplug can be done asynchronously. Thus, sending the second device_del before the previous unplug is complete may lead to unexpected results. On PCIe devices, this cancels the hot-unplug process. Signed-off-by: Julia Suvorova <jusual@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20200220165556.39388-1-jusual@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-25mem-prealloc: optimize large guest startupbauerchen
[desc]: Large memory VM starts slowly when using -mem-prealloc, and there are some areas to optimize in current method; 1、mmap will be used to alloc threads stack during create page clearing threads, and it will attempt mm->mmap_sem for write lock, but clearing threads have hold read lock, this competition will cause threads createion very slow; 2、methods of calcuating pages for per threads is not well;if we use 64 threads to split 160 hugepage,63 threads clear 2page,1 thread clear 34 page,so the entire speed is very slow; to solve the first problem,we add a mutex in thread function,and start all threads when all threads finished createion; and the second problem, we spread remainder to other threads,in situation that 160 hugepage and 64 threads, there are 32 threads clear 3 pages,and 32 threads clear 2 pages. [test]: 320G 84c VM start time can be reduced to 10s 680G 84c VM start time can be reduced to 18s Signed-off-by: bauerchen <bauerchen@tencent.com> Reviewed-by: Pan Rui <ruippan@tencent.com> Reviewed-by: Ivan Ren <ivanren@tencent.com> [Simplify computation of the number of pages per thread. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-25memory: batch allocate ioeventfds[] in address_space_update_ioeventfds()Stefan Hajnoczi
Reallocing the ioeventfds[] array each time an element is added is very expensive as the number of ioeventfds increases. Batch allocate instead to amortize the cost of realloc. This patch reduces Linux guest boot times from 362s to 140s when there are 2 virtio-blk devices with 1 virtqueue and 99 virtio-blk devices with 32 virtqueues. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20200218182226.913977-1-stefanha@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-24Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell
staging Pull request This pull request contains a virtio-blk/scsi performance optimization, event loop scalability improvements, and a qtest-based device fuzzing framework. I am including the fuzzing patches because I have reviewed them and Thomas Huth is currently away on leave. # gpg: Signature made Sat 22 Feb 2020 08:50:05 GMT # gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full] # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full] # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: (31 commits) fuzz: add documentation to docs/devel/ fuzz: add virtio-scsi fuzz target fuzz: add virtio-net fuzz target fuzz: add i440fx fuzz targets fuzz: add configure flag --enable-fuzzing fuzz: add target/fuzz makefile rules fuzz: add support for qos-assisted fuzz targets fuzz: support for fork-based fuzzing. main: keep rcu_atfork callback enabled for qtest exec: keep ram block across fork when using qtest fuzz: add fuzzer skeleton libqos: move useful qos-test funcs to qos_external libqos: split qos-test and libqos makefile vars libqos: rename i2c_send and i2c_recv qtest: add in-process incoming command handler libqtest: make bufwrite rely on the TransportOps libqtest: add a layer of abstraction to send/recv qtest: add qtest_server_send abstraction fuzz: add FUZZ_TARGET module type module: check module wasn't already initialized ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-22fuzz: add documentation to docs/devel/Alexander Bulekov
Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-23-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add virtio-scsi fuzz targetAlexander Bulekov
The virtio-scsi fuzz target sets up and fuzzes the available virtio-scsi queues. After an element is placed on a queue, the fuzzer can select whether to perform a kick, or continue adding elements. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-22-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add virtio-net fuzz targetAlexander Bulekov
The virtio-net fuzz target feeds inputs to all three virtio-net virtqueues, and uses forking to avoid leaking state between fuzz runs. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-21-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add i440fx fuzz targetsAlexander Bulekov
These three targets should simply fuzz reads/writes to a couple ioports, but they mostly serve as examples of different ways to write targets. They demonstrate using qtest and qos for fuzzing, as well as using rebooting and forking to reset state, or not resetting it at all. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-20-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add configure flag --enable-fuzzingAlexander Bulekov
Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-19-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add target/fuzz makefile rulesAlexander Bulekov
Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20200220041118.23264-18-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add support for qos-assisted fuzz targetsAlexander Bulekov
Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-17-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: support for fork-based fuzzing.Alexander Bulekov
fork() is a simple way to ensure that state does not leak in between fuzzing runs. Unfortunately, the fuzzer mutation engine relies on bitmaps which contain coverage information for each fuzzing run, and these bitmaps should be copied from the child to the parent(where the mutation occurs). These bitmaps are created through compile-time instrumentation and they are not shared with fork()-ed processes, by default. To address this, we create a shared memory region, adjust its size and map it _over_ the counter region. Furthermore, libfuzzer doesn't generally expose the globals that specify the location of the counters/coverage bitmap. As a workaround, we rely on a custom linker script which forces all of the bitmaps we care about to be placed in a contiguous region, which is easy to locate and mmap over. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-16-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22main: keep rcu_atfork callback enabled for qtestAlexander Bulekov
The qtest-based fuzzer makes use of forking to reset-state between tests. Keep the callback enabled, so the call_rcu thread gets created within the child process. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20200220041118.23264-15-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22exec: keep ram block across fork when using qtestAlexander Bulekov
Ram blocks were marked MADV_DONTFORK breaking fuzzing-tests which execute each test-input in a forked process. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-14-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22fuzz: add fuzzer skeletonAlexander Bulekov
tests/fuzz/fuzz.c serves as the entry point for the virtual-device fuzzer. Namely, libfuzzer invokes the LLVMFuzzerInitialize and LLVMFuzzerTestOneInput functions, both of which are defined in this file. This change adds a "FuzzTarget" struct, along with the fuzz_add_target function, which should be used to define new fuzz targets. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-13-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22libqos: move useful qos-test funcs to qos_externalAlexander Bulekov
The moved functions are not specific to qos-test and might be useful elsewhere. For example the virtual-device fuzzer makes use of them for qos-assisted fuzz-targets. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-12-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22libqos: split qos-test and libqos makefile varsAlexander Bulekov
Most qos-related objects were specified in the qos-test-obj-y variable. qos-test-obj-y also included qos-test.o which defines a main(). This made it difficult to repurpose qos-test-obj-y to link anything beside tests/qos-test against libqos. This change separates objects that are libqos-specific and ones that are qos-test specific into different variables. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20200220041118.23264-11-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22libqos: rename i2c_send and i2c_recvAlexander Bulekov
The names i2c_send and i2c_recv collide with functions defined in hw/i2c/core.c. This causes an error when linking against libqos and softmmu simultaneously (for example when using qtest inproc). Rename the libqos functions to avoid this. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Acked-by: Thomas Huth <thuth@redhat.com> Message-id: 20200220041118.23264-10-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22qtest: add in-process incoming command handlerAlexander Bulekov
The handler allows a qtest client to send commands to the server by directly calling a function, rather than using a file/CharBackend Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-9-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22libqtest: make bufwrite rely on the TransportOpsAlexander Bulekov
When using qtest "in-process" communication, qtest_sendf directly calls a function in the server (qtest.c). Previously, bufwrite used socket_send, which bypasses the TransportOps enabling the call into qtest.c. This change replaces the socket_send calls with ops->send, maintaining the benefits of the direct socket_send call, while adding support for in-process qtest calls. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-8-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22libqtest: add a layer of abstraction to send/recvAlexander Bulekov
This makes it simple to swap the transport functions for qtest commands to and from the qtest client. For example, now it is possible to directly pass qtest commands to a server handler that exists within the same process, without the standard way of writing to a file descriptor. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Message-id: 20200220041118.23264-7-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>