README
######

Refactoring job layouts
=======================

0: YAML supports comments, so all jobs in this repository need to use
   comments to explain why the jobs are as they are.
   * Include details of why these files are used and not others.
   * Include details of known problems - like an outdated kernel which
     does not support a particular rootfs OS.
   * Use standard images from images.validation.linaro.org/snapshots.linaro.org/components/lava/standard
     wherever possible. Link to the README produced by the standard build
     everytime a standard build is used.
   * Comment liberally and be verbose but keep the comments relevant.
     Avoid the "Sample job for" top level comment.

1: Consistent structure of YAML
   * Avoid shorthand syntax like ['blah', 'foo'] to make it easier to
     add or update or comment on particular settings later. Lists and
     dictionaries need to be multi-line.
   * Highlight if an item is a list of dictionaries rather than just 
     a list or just a dictionary, other than the list of action
     dictionaries.
   * Put the device_type and job_name at the top of the file
   * Make the job_name unique and meaningful but do not make it so long
     as to be hard to remember or understand. Use comments to clarify
     what the job actually does.
   * Timeouts, priority and visibility near the top of the file, before
     the actions list of dictionaries.
   * Jobs are to be priority medium and public visibility, unless
     the device_type or image location requires otherwise.
   * Be consistent in the ordering within dictionaries. It does not
     matter to YAML or to LAVA but it helps readers understand the job.
     e.g. when using lava-multinode protocol, put role list at the top of
     the action stanza:

  - deploy:
     role:
     - client
     - server
     timeout:
       minutes: 10
     to: tftp

2: Use local/ for your own tests which involve SSH public keys or other
   identifiers. local/ is listed in .gitignore.

3: Use standard/ for jobs which both use standard builds *and* are
   sufficiently stable that the job itself can be considered as a
   standard job for others to use as a base for comparison.

4: Multinode
   * Always use inline definitions for calls from a test shell to the
     Multinode API. This will support later work on describing the
     multinode synchronisation flow in the UI.

5: Documentation
   * If there are particular issues which are common to a range of jobs
     or all jobs of a device type or all jobs using a particular
     protocol, add .rst documentation in docs.

6: Avoid simply copying failure_retry into every job. A lot of the early
   jobs have this due to early copying of unit test job files but it is
   not necessary in most cases. Where failure_retry is needed or useful,
   add a comment specifying why.

7: Make test names useful so that queries and charts can be made more
   easily.
   * Avoid combining the job_name into the test name - let the test name
     relate to the test definitions.
   * The top level name for the test is not currently visible in the
     UI, so omit.
8: timeouts
   * Make timeouts small with some margin so that these jobs will only
     timeout when something is actually wrong on the instance.
9: Avoid local configuration - the expectation is that these jobs will
   be submitted to labs like Cambridge with PDU support. Extra support
   for shutdown-message parameters can be kept as copies of files in
   local/.
10: Keep consistent indenting so that blocks can be copied and pasted
    between jobs. Validate all jobs using the online YAML parser 
    http://yaml-online-parser.appspot.com/?yaml=&type=json before
    commit. Make it easy to compare definitions.