Added documentation on how to deploy and use WAv2-in-LAVA

2 files changed, 125 insertions, 89 deletions

diff --git a/README b/README
new file mode 100644
index 0000000..e57f1d5
--- /dev/null
+++ b/README
@@ -0,0 +1,84 @@
+Workload Automation v2 in LAVA (WAv2)
+*************************************
+This directory contains ... TODO
+
+Job Construction
+================
+Coordination of the workload automation *host* and the Versatile Express TC2 *target* is done through a multi-node job definition (wav2multi.json).  The actions in the JSON file are assigned to specific roles (host & target).  The roles are then invoked using lava ACTIONs and in these jobs, the roles execute respective lava-test-shell tests defined in YAML (wa2host.yaml) and wa2target.yaml).   The two devices coordinate their execution using synchronization API functions within the lava-test-shell tests.  In this test there are 3 api calls used.
+
+* **lava-wait** (waits for data from **lava-send**)
+* **lava-sync** (waits for all devices)
+
+Parameter description for wa2host.yaml, wa2target.yaml and wa2host_postprocessing.yaml
+* wa2host.yaml
+ * AGENDA - name of the agenda file to run specific workload on TC2. Agenda files are stored in this repository in templates directory.
+ * JOB_NAME - name that identifies job or set of jobs that aggregate results in a single database
+ * MODE - mp (HMP mode booting to A7 cluster), iks (In Kernel Switcher), a7only (only A7 cluster online), a15only (only A15 cluster online)
+ * SIGNAL_PREFIX - identifies the host-target pair of lava-test-shells
+* wa2target.yaml
+ * SIGNAL_PREFIX - identifies the host-target pair of lava-test-shells
+* wa2host_postprocessing.yaml
+ * JOB_NAME - name that identifies job or set of jobs that aggregate results in a single database. Postprocessing YAML uses this database
+ * DELETE_DB - if set to true, database is deleted after postprocessing is done. Default: false
+
+More information on multi-node can be found in the LAVA documentation:
+
+* Documentation:  https://validation.linaro.org/static/docs/writing-multinode.html
+* API: https://validation.linaro.org/static/docs/multinodeapi.html?highlight=multinode
+
+Data postprocessing
+===================
+To simplify the post processing of data from multiple lava actions and possibly multiple lava test runs, the test uses an external sqlite database (hosted on the Xen KVM) and the WAv2  options to store the results until they are ready to be processed (after the entire WAv2 run). Database schema and contents are created by WAv2. Database is located in /root/db_results/ directory. Database name is defined in wa2host.yaml as a parameter JOB_NAME. The parameter has to have the same value for all the jobs running on the same build in order to store the results in the same database. 
+
+Database data postprocessing is done using separate YAML definition - wa2host_postprocessing.yaml. It takes one parameter that defines the name of the database to use. The parameter name is the same as for testing jobs: JOB_NAME 
+
+Overview of LAVA Architecture & Security
+========================================
+Due to licensing, WAv2-in-LAVA uses capabilities that restrict access to a specific user or group in LAVA.
+
+Job submission security
+-----------------------
+Only authorized users are allowed to submit jobs onto the protected device.  For the WAv2 testing, the protected device in LAVA is an SSH connection onto a Xen server in the lab that contains the workload automation secure repository, see diagram below.
+
+Results security
+----------------
+The test jobs we execute in LAVA submit results into private bundle streams with restricted access controls.
+
+Security block diagram
+----------------------
+::
+
+    wa2multi.json
+   +---------------------------------------------------------+
+   |  wa2host.yaml                   wa2target.yaml          |
+   | +-----------------------+      +----------------------+ |
+   | |host: dummy-ssh        |      |target: vexpress-tc2  | |
+   | | Xen server            |      |deploy: android (lsk) | |
+   | |    +WAv2 Repository   |  IP  |                      | |
+   | |                       |<-----|                      | |
+   | |restricted to NDA list |      |                      | |
+   | +----|------------------+      +----------------------+ |
+   +------|--------------------------------------------------+
+          v
+    +---------------------+
+    | Xen Server          |
+    | + WAv2 Repository   |
+    +---------------------+
+
+Xen Server Configuration
+========================
+The Xen server is a Linux server with the licensed WAv2 repository installed.  LAVA connects to the Xen server over an SSH connection using a dummy-ssh device-type in LAVA[1].  
+
+The Xen server is a *persistent* device under control in LAVA, which means that data is persistent over multiple test runs and that only 1 LAVA test can run at a time to prevent accidental access collisions.  In an environment that has multiple WAv2 setups, you would need to scale up your WAv2 *host* architecture.  
+
+Filesystem & software Requirements:
+
+* /root/templates - contents
+  WAv2-in-LAVA uses adb over ethernet. WAv2 supports adb over ethernet since version 2.1.4. In order to point WAv2 to correct device config.py file needs to have correct contents. Since config.py is a part of WAv2 and is restricted by the license, it is not available as a part of this git repository. config.py template has to be located in /root/templates/ directory. The template has to have entry for adb_name as $ipaddr. The template is copied to the job directory when LAVA job is started on host and $ipaddr is replaced with correct value of targer IP address.
+* WAv2 installed - TODO
+* other? TODO
+
+WAv2 uses the LAVA dummy-ssh device-type, an SSH connection to an external device that can be controlled by LAVA.  
+For more information, see the documentation at:  
+
+* dummy-ssh: https://validation.linaro.org/static/docs/dummy-deploy.html#configuration-dummy-ssh
diff --git a/wa2multi.json b/wa2multi.json
index d2e9bc4..234377f 100644
--- a/wa2multi.json
+++ b/wa2multi.json
@@ -2,7 +2,7 @@
     "health_check": false,
     "logging_level": "INFO",
     "timeout": 18000,
-    "job_name": "WAv2 HMP audio, bbench+audio, 5 iterations",
+    "job_name": "WAv2 audio in MP, IKS, A15, A7 modes, 2 iterations each",
     "device_group": [
         {
             "role": "host",
@@ -45,7 +45,7 @@
                         "testdef": "wa2host.yaml",
                         "parameters": {
                             "AGENDA": "hmp_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
+                            "JOB_NAME": "lsk_207",
                             "MODE": "mp",
                             "SIGNAL_PREFIX": "hmp-audio-iter1"
                         }
@@ -81,7 +81,7 @@
                         "testdef": "wa2host.yaml",
                         "parameters": {
                             "AGENDA": "hmp_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
+                            "JOB_NAME": "lsk_207",
                             "MODE": "mp",
                             "SIGNAL_PREFIX": "hmp-audio-iter2"
                         }
@@ -101,6 +101,7 @@
                         "parameters": {
                             "SIGNAL_PREFIX": "hmp-audio-iter2"
                         }
+
                     }
                 ],
                 "timeout": 2400,
@@ -115,45 +116,10 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2host.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-audio-iter3"
-                        }
-                    }
-                ],
-                "timeout": 4200,
-                "role": "host"
-            }
-        },
-        {
-            "command": "lava_test_shell",
-            "parameters": {
-                "testdef_repos": [
-                    {
-                        "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
-                        "testdef": "wa2target.yaml",
-                        "parameters": {
-                            "SIGNAL_PREFIX": "hmp-audio-iter3"
-                        }
-                    }
-                ],
-                "timeout": 2400,
-                "role": "target"
-            }
-        },
-        {
-            "command": "lava_test_shell",
-            "parameters": {
-                "testdef_repos": [
-                    {
-                        "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
-                        "testdef": "wa2host.yaml",
-                        "parameters": {
-                            "AGENDA": "hmp_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-audio-iter4"
+                            "AGENDA": "iks_audio.yaml",
+                            "JOB_NAME": "lsk_207",
+                            "MODE": "iks",
+                            "SIGNAL_PREFIX": "iks-audio-iter1"
                         }
                     }
                 ],
@@ -169,8 +135,9 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2target.yaml",
                         "parameters": {
-                            "SIGNAL_PREFIX": "hmp-audio-iter4"
+                            "SIGNAL_PREFIX": "iks-audio-iter1"
                         }
+
                     }
                 ],
                 "timeout": 2400,
@@ -185,10 +152,10 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2host.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-audio-iter5"
+                            "AGENDA": "iks_audio.yaml",
+                            "JOB_NAME": "lsk_207",
+                            "MODE": "iks",
+                            "SIGNAL_PREFIX": "iks-audio-iter2"
                         }
                     }
                 ],
@@ -204,8 +171,9 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2target.yaml",
                         "parameters": {
-                            "SIGNAL_PREFIX": "hmp-audio-iter5"
+                            "SIGNAL_PREFIX": "iks-audio-iter2"
                         }
+
                     }
                 ],
                 "timeout": 2400,
@@ -220,10 +188,10 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2host.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_bbench_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter1"
+                            "AGENDA": "a7only_audio.yaml",
+                            "JOB_NAME": "lsk_207",
+                            "MODE": "a7only",
+                            "SIGNAL_PREFIX": "a7only-audio-iter1"
                         }
                     }
                 ],
@@ -239,7 +207,7 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2target.yaml",
                         "parameters": {
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter1"
+                            "SIGNAL_PREFIX": "a7only-audio-iter1"
                         }
 
                     }
@@ -256,10 +224,10 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2host.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_bbench_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter2"
+                            "AGENDA": "a7only_audio.yaml",
+                            "JOB_NAME": "lsk_207",
+                            "MODE": "a7only",
+                            "SIGNAL_PREFIX": "a7only-audio-iter2"
                         }
                     }
                 ],
@@ -275,8 +243,9 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2target.yaml",
                         "parameters": {
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter2"
+                            "SIGNAL_PREFIX": "a7only-audio-iter2"
                         }
+
                     }
                 ],
                 "timeout": 2400,
@@ -291,10 +260,10 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2host.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_bbench_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter3"
+                            "AGENDA": "a15only_audio.yaml",
+                            "JOB_NAME": "lsk_207",
+                            "MODE": "a15only",
+                            "SIGNAL_PREFIX": "a15only-audio-iter1"
                         }
                     }
                 ],
@@ -310,8 +279,9 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2target.yaml",
                         "parameters": {
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter3"
+                            "SIGNAL_PREFIX": "a15only-audio-iter1"
                         }
+
                     }
                 ],
                 "timeout": 2400,
@@ -326,10 +296,10 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2host.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_bbench_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter4"
+                            "AGENDA": "a15only_audio.yaml",
+                            "JOB_NAME": "lsk_207",
+                            "MODE": "a15only",
+                            "SIGNAL_PREFIX": "a15only-audio-iter2"
                         }
                     }
                 ],
@@ -345,8 +315,9 @@
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
                         "testdef": "wa2target.yaml",
                         "parameters": {
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter4"
+                            "SIGNAL_PREFIX": "a15only-audio-iter2"
                         }
+
                     }
                 ],
                 "timeout": 2400,
@@ -359,12 +330,9 @@
                 "testdef_repos": [
                     {
                         "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
-                        "testdef": "wa2host.yaml",
+                        "testdef": "wa2host_postprocessing.yaml",
                         "parameters": {
-                            "AGENDA": "hmp_bbench_audio.yaml",
-                            "JOB_NAME": "hmp_multinode_experiment1",
-                            "MODE": "mp",
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter5"
+                            "JOB_NAME": "lsk_207"
                         }
                     }
                 ],
@@ -373,22 +341,6 @@
             }
         },
         {
-            "command": "lava_test_shell",
-            "parameters": {
-                "testdef_repos": [
-                    {
-                        "git-repo": "git://git.linaro.org/people/milosz.wasilewski/wa2-lava.git",
-                        "testdef": "wa2target.yaml",
-                        "parameters": {
-                            "SIGNAL_PREFIX": "hmp-bbench-audio-iter5"
-                        }
-                    }
-                ],
-                "timeout": 2400,
-                "role": "target"
-            }
-        },
-        {
             "command": "submit_results_on_host",
             "parameters": {
                 "stream": "/anonymous/mwasilew/",