automated/linux/ptest: Analyze each test in package tests
Currently ptest.py analyze only exit code of each package test
to decide if it passed or not. However, ptest-runner can return
success code even though some tests failed. So we need to parse
test output and analyze it.
It also quite useful to see exactly which tests failed. So results are
recorded for each particular test, and lava-test-set feature is used
to distinguish packages.
Test results,
https://validation.linaro.org/results/1900356
automated/utils/send-to-lava.sh: Add support to send lava-test-set results
Change-Id: Id1c2e8190b9cce5cfa9f08e64322667f8e1a0c28
Signed-off-by: Oleksandr Terentiev <oterenti@cisco.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
diff --git a/automated/utils/send-to-lava.sh b/automated/utils/send-to-lava.sh
index bf2a477..fdfdf78 100755
--- a/automated/utils/send-to-lava.sh
+++ b/automated/utils/send-to-lava.sh
@@ -4,6 +4,8 @@
which lava-test-case > /dev/null 2>&1
lava_test_case="$?"
+which lava-test-set > /dev/null 2>&1
+lava_test_set="$?"
if [ -f "${RESULT_FILE}" ]; then
while read -r line; do
@@ -31,6 +33,18 @@
else
echo "<TEST_CASE_ID=${test} RESULT=${result} MEASUREMENT=${measurement} UNITS=${units}>"
fi
+ elif echo "${line}" | egrep -iq "^lava-test-set.*"; then
+ test_set_status="$(echo "${line}" | awk '{print $2}')"
+ test_set_name="$(echo "${line}" | awk '{print $3}')"
+ if [ "${lava_test_set}" -eq 0 ]; then
+ lava-test-set "${test_set_status}" "${test_set_name}"
+ else
+ if [ "${test_set_status}" = "start" ]; then
+ echo "<LAVA_SIGNAL_TESTSET START ${test_set_name}>"
+ else
+ echo "<LAVA_SIGNAL_TESTSET STOP>"
+ fi
+ fi
fi
done < "${RESULT_FILE}"
else