LTP: Convert LKFT skipfiles to a yaml skipfile
skipfile-lkft.yaml encompasses all of the lkft skipfiles in this
directory as well as the skipfiles/ directory. They may be removed once
build jobs are updated to use a generated skipfile.
To generate a skipfile from skipfile-lkft.yaml, use skipgen, which is
included in this repository at /bin/<arch>/skipgen. ltp.sh will be
expanded to support skipgen separately.
The actual contents have been derived from the various skipfiles. In
cases of inconsistency, the skip was expanded. For example,
netns_breakns_ip_ipv4_ioctl is currently skipped on 4.4 hikey, 4.4 juno,
and all branches of x15 and x86. In skipfile-lkft.yaml, it's skipped for
all branches on all boards.
No other modifications to the skipfiles have been implemented. There are
obvious skips that need adjusting, but that will be done subsequently.
This review is just a one to one translation of the skipfiles into one
yaml skipfile. If you notice things that are wrong or shouldn't be
listed anymore - good! That's the point of this effort 😀
Lastly, teach validate.py about skipgen yaml files. Validate them by
running skipgen itself against them and verifying that a skiplist of at
least 1 skip is produced.
Change-Id: Ic46af16d36dfd30ed0876808f916318cddd44fa2
Signed-off-by: Dan Rue <dan.rue@linaro.org>
diff --git a/validate.py b/validate.py
index 3f127c4..13a4a65 100755
--- a/validate.py
+++ b/validate.py
@@ -32,6 +32,14 @@
print_stderr(result_message)
+def detect_abi():
+ # Retrieve the current canonical abi from
+ # automated/lib/sh-test-lib:detect_abi
+ return subprocess.check_output(
+ ". automated/lib/sh-test-lib && detect_abi && echo $abi",
+ shell=True).decode('utf-8').strip()
+
+
def pep8_check(filepath, args):
_fmt = "%(row)d:%(col)d: %(code)s %(text)s"
options = {
@@ -62,17 +70,9 @@
return 0
-def metadata_check(filepath, args):
- if filepath.lower().endswith("yaml"):
- filecontent = None
- try:
- with open(filepath, "r") as f:
- filecontent = f.read()
- except FileNotFoundError:
- publish_result(["* METADATA [PASSED]: " + filepath + " - deleted"], args)
- return 0
+def validate_yaml_contents(filepath, args):
+ def validate_lava_yaml(y, args):
result_message_list = []
- y = yaml.load(filecontent)
if 'metadata' not in y.keys():
result_message_list.append("* METADATA [FAILED]: " + filepath)
result_message_list.append("\tmetadata section missing")
@@ -102,7 +102,37 @@
return 1
result_message_list.append("* METADATA [PASSED]: " + filepath)
publish_result(result_message_list, args)
- return 0
+ return 0
+
+ def validate_skipgen_yaml(filepath, args):
+ abi = detect_abi()
+ # Run skipgen on skipgen yaml file to check for output and errors
+ skips = subprocess.check_output(
+ "automated/bin/{}/skipgen {}".format(abi, filepath),
+ shell=True).decode('utf-8').strip()
+ if len(skips.split('\n')) < 1:
+ publish_result(["* SKIPGEN [FAILED]: " + filepath + " - No skips found"], args)
+ return 1
+ publish_result(["* SKIPGEN [PASSED]: " + filepath], args)
+ return 0
+
+ filecontent = None
+ try:
+ with open(filepath, "r") as f:
+ filecontent = f.read()
+ except FileNotFoundError:
+ publish_result(["* YAMLVALIDCONTENTS [PASSED]: " + filepath + " - deleted"], args)
+ return 0
+ y = yaml.load(filecontent)
+ if 'metadata' in y.keys():
+ # lava yaml file
+ return validate_lava_yaml(y, args)
+ elif 'skiplist' in y.keys():
+ # skipgen yaml file
+ return validate_skipgen_yaml(filepath, args)
+ else:
+ publish_result(["* YAMLVALIDCONTENTS [FAILED]: " + filepath + " - Unknown yaml type detected"], args)
+ return 1
def validate_yaml(filename, args):
@@ -168,7 +198,7 @@
exitcode = validate_yaml(path, args)
if exitcode == 0:
# if yaml isn't valid there is no point in checking metadata
- exitcode = metadata_check(path, args)
+ exitcode = validate_yaml_contents(path, args)
elif run_pep8 and path.endswith(".py"):
exitcode = pep8_check(path, args)
elif path.endswith(".php"):