diff options
| author | Christophe Lyon <christophe.lyon@linaro.org> | 2017-06-01 14:48:26 +0000 |
|---|---|---|
| committer | Christophe Lyon <christophe.lyon@linaro.org> | 2017-06-02 06:52:54 +0000 |
| commit | 01e8914b788ab5363ae7ed7e250322ff4936069f (patch) | |
| tree | c60390906488dca77f399fea3f2743a453235f59 /tcwg-compare-results.yaml | |
| parent | 995997dbb0b7559205cdc7c55aa945883564f8c9 (diff) | |
tcwg-compare-results: Add pass_thresh parameter.
tcwg-trigger-binutils: Set pass_thresh to 0.92.
Some binutils confguration have a low pass rate (about 0.92), because
of many untested/unsupported tests. This causes the non-regression
comparison to return an error even if there was no regression, because
the default minimum expected pass rate is 0.95.
Use the new -pass-thresh comparison parameter, to override the value
for binutils and set it to 0.92 to avoid reporting errors all the
time.
Change-Id: I0174079b73fb56a3ed3252b04329c50b93f52e7b
Diffstat (limited to 'tcwg-compare-results.yaml')
| -rw-r--r-- | tcwg-compare-results.yaml | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/tcwg-compare-results.yaml b/tcwg-compare-results.yaml index f781ca65a9..bdb6cd4ce2 100644 --- a/tcwg-compare-results.yaml +++ b/tcwg-compare-results.yaml @@ -24,6 +24,10 @@ default: buildfarm/gcc-linaro-5-branch description: 'Log directory on $log_server' - string: + name: pass_thresh + default: '0.95' + description: 'Pass threshold, below which the comparison returns an error' + - string: name: log_server default: dev-01.tcwglab:/home/tcwg-buildslave/logs description: 'Log_Server' @@ -80,6 +84,7 @@ ./compare_dg_tests.pl \ ./unstable-tests.txt ${log_server}:${dest} || status=1 ssh ${log_server} bash ${dest}/compare_jobs.sh \ + -pass-thresh ${pass_thresh} \ $basedir/$ref_logs $basedir/$new_logs || status=1 # Copy the generated reports |