Release notes ============= 1. Preface a. Proprietary notice Copyright (c) 2012, ARM Limited All rights reserved. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. b. License details Copyright (c) 2009-2012, ARM Limited. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of ARM nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. c. Document confidentiality status Redistribution of source and compiled code is subject to the license terms above. d. Product status ARM Virtualizer for Cortex-A15/Cortex-A7 Task Migration v2.4 e. Web address Not applicable. f. Feedback on the ARM Virtualizer software None at present. g. Feedback on this release note document None at present. 2. Release details a. Product release status v2.4 b. ARM Virtualizer software release v2.4 This software release is a v2.4 snapshot of the ARM Virtualizer software. The ARM Virtualizer software is example code that demonstrates cluster context switching capability on a coherent dual cluster system composed of a Cortex-A7 cluster and a Cortex-A15 cluster. The intent behind this delivery is to allow inspection of the software architecture and to demonstrate existing functionality. It is possible to execute the ARM Virtualizer software on a Real-Time System Model (RTSM_VE_Cortex_A15x1_A7x1 and RTSM_VE_Cortex_A15x4_A7x4). This model may be obtained from ARM by separate arrangement. c. Deliverables This release contains the following file: 1. arm-virtualizer-v2_4-170512.tar.bz2 - Contains source code for a basic boot wrapper. This boot wrapper performs minimal system initialization and boots the system with the Virtualizer code. It also permits booting the system with an optional Linux kernel image and an accompanying root filesystem [both NOT supplied with this release]. - Contains source code for the ARM Virtualizer software. - Contains pertinent documentation covering the release components, their installation and usage. d. Functionality included This release of the ARM Virtualizer software is capable of rapid and robust cluster context switching on a coherent system between a cluster of up to four Cortex-A7 processors and a cluster of up to four Cortex-A15 processors. In addition, this release of the ARM Virtualizer software permits payload software (bare-metal software or a Linux operating system kernel) built for the Cortex-A15 processor cluster to run un-modified on a Cortex-A7 processor cluster. e. New features 1. Context of the registers specified by the v7.1 Debug architecture is saved and restored during a cluster context switch operation using the mandatory CP14 interface instead of the optional memory mapped interface. Please note that it is not possible to set hardware breakpoints on v7.0 of the big.LITTLE FastModels. This functionality is expected to be available in v7.1 of the FastModels. 2. It is possible to use the Cortex-A7 MP4 processor cluster as the boot cluster provided it is also configured as the target cluster. Refer to "docs/01-Usage.txt" for details. 3. A new API is used to request services from the Virtualizer using the ARM "HVC #0" instruction. Details of this API are documented in "docs/09-HVC-calling-conventions.pdf". Existing services which have been changed to adhere to the new standard are: 1. HVC to invoke a cluster switch 2. HVC to read the physical MPIDR coprocessor register New services which have been implemented using the new standard are: 1. Services to enable use of the PMU with the Virtualizer. Details can be found in: "docs/10-ARM-Virtualizer-support-for-debug-and-the-PMU.pdf" f. Known issues 1. This release is intended to be built in a Linux development environment. Environments other than Linux are not supported. 2. The v7.0 release of the FastModels does not reset DBGOSDLR.DLK to 0 when a processor is brought out of warm reset. A software workaround has been implemented which clears the DLK field as a part of saving and restoring the context. 3. Cortex-A15 implements 6 event counters while Cortex-A7 implements 4. The Virtualizer hides this difference by restricting the number of counters visible on Cortex-A15 to 4. Implementing this requires the PMCR.N field to mirror the HDCR.HPMN field. v7.0 of the FastModels are unable to virtualize the PMCR.N field using this method. Hence Counter #5 & #6 should not be used by the payload software. This issue is expected to be fixed in a subsequent release of the FastModels. g. Issues resolved since last release 1. Bug fixes v2.3 to v2.4: 1. It is possible to boot the payload software on the Cortex-A7 cluster. This means that software built for Cortex-A7 can be run unmodified on the Cortex-A15 cluster. 2. Cortex-A15 has a PIPT I-cache with 64 byte cache lines. Cortex-A7 has a VIPT I-cache with 32 byte cache lines. To provide a uniform view of the I-cache topology to the payload software, the Virtualizer exports the Cortex-A7 I-cache topology to Cortex-A15, thus hiding the differences between them. 3. The cache level chosen through a write to the CSSELR on the Cortex-A7 cluster is now migrated to the Cortex-A15 cluster during a subsequent migration. 4. Support added in v2.3 for migrating virtual interrupts as a part of physical interrupt migration required the following fixes: 1. Event for acknowledging transfer of virtual interrupts is now propagated correctly. 2. Data structure used for enqueuing a virtual interrupt is reset correctly. 5. The pre-built stub image used for synchronous cluster switching now prints out the outbound cluster ID and CPU ID. [See docs/01-Usage.txt for details]. v2.2 to v2.3: 1. When a physical interrupt is migrated from one cpu interface to another on any cluster, it is possible that its virtual interrupt is in a pending state in the HYP view interface list registers. It is now ensured that the virtual interrupt is also migrated by requesting it to be added to the queue of virtual interrupts on the destination cpu interface. v2.1 to v2.2: 1. Issue an DCCISW and not a DCCSW when DCCSIW was trapped in trap_cp15_mrc_mcr_handle (big-little/virtualisor/virt_handle.c). A DCCISW (Data Cache clean and invalidate by set/way) operation was being incorrectly implemented as a DCCSW (Data Cache clean by set/way) operation by the Virtualizer. 2. Calculate l2_desc correctly in CreateL3PageTable (big-little/common/pagetable_setup.c) for level equals 2. Correct index into Level-2 page table of the 2nd Stage translations is being calculated when CreateL3PageTable() is called to add a L3 page table for IPAs > 1GB. 3. Create second 4KB 2nd stage mapping for VGIC in Create2ndStagePageTables (big-little/common/pagetable_setup.c). Mapping added to 2nd stage translation tables to cover the 2nd 4K memory map of the physical cpu interface of the vGIC (cpu interface base + 0x1000). 4. Fix race condition during enabling CCI coherency in warm_reset (big-little/secure_world/monmode_vectors.s). A race condition existed while enabling CCI coherency after a warm reset in the Secure world code. This could have led to a case where cpus1-3 start restoring the saved context without taking part in CCI based coherency for a brief period of time. v2.1: 1. vGIC HYP view interface handling code in (common/vgiclib.c) now detects the number of implemented list registers from the vgic type register instead of assuming that the maximum (64) will be present. h. Test cases and results In accordance with the delivery's status as example code, testing is sufficient to prove robustness of the basic implementation but does not provide full coverage for all use cases. 1. This release has been tested for correct cluster switching operation at ~12 million cycle switching intervals with bare-metal and Linux kernel payloads. 2. This release has been tested using a select subset of an ARM internal Linux based stress testing suite. 3. The Linux Test Project (LTP) cpu hotplug tests have been run while cluster switching was performed simultaneously. All tests passed. i. Other information Not applicable. 3. Documentation The docs subdirectory contains the following notes: 01-Usage.txt: General installation and usage instructions. 02-Code-layout.txt: Overview of the code layout. 03-Linux-kernel-build.txt: Instructions on obtaining and building a Linux kernel image suitable for the virtualizer. 04-Cache-hit-rate-howto.txt: Description of the MTI trace plugin infrastructure and ways to use the trace for estimating snoop hit rates across cluster switches. 05-FAQ.txt: Placeholder for commonly asked questions with answers. 06-Optional-rootfs-build.txt: Instructions for building and using rootfilesystems with the virtualizer. 07-Linux-cpu-hotplug-howto.txt: Instructions & guidelines to use Linux CPU hotplug. 08-Streamline-and-cluster-switching-howto.txt: Instructions & guidelines to use ARM Streamline tools with the Virtualizer software. "09-HVC-calling-conventions.pdf": Describes a set of software conventions for Hyper-calls to be used with the ARM virtualization extensions. "10-ARM-Virtualizer-support-for-debug-and-the-PMU.pdf": Describes the Performance Monitor Units on a big.LITTLE system and the interface to these units as provided by the Virtualizer software. 4. Tools a. Tools 1. ARM Development Studio 5 - Version 5.9. 2. Real-Time System Model v7.0.1 (RTSM_VE_Cortex_A15x1_A7x1 and RTSM_VE_Cortex_A15x4_A7x4). b. Operating systems 1. Ubuntu 10.10. 2. Red Hat Enterprise Linux WS release 4 (Nahant Update 4). 5. Support ARM provides this software release purely as example software and absolves itself of any support burden. ARM is open to receiving any constructive feedback and at it's discretion, will endeavour to incorporate any feedback in subsequent releases of this example software.