blob: e69ddab6f0ed07a35dfaa8a762339d144a95dab7 [file] [log] [blame]
Comparing:
REFERENCE:sum-1.txt
CURRENT: sum-2.txt
+---------+---------+
o RUN STATUS: | REF | RES |
+------------------------------------------+---------+---------+
| Passes [PASS] | 22 | 16 |
| Unexpected fails [FAIL] | 11 | 12 |
| Errors [ERROR] | 1 | 1 |
| Unexpected passes [XPASS] | 9 | 11 |
| Expected fails [XFAIL] | 9 | 11 |
| Known fails [KFAIL] | 9 | 11 |
| Unresolved [UNRESOLVED] | 9 | 10 |
| Unsupported [UNSUPPORTED] | 9 | 10 |
| Untested [UNTESTED] | 9 | 10 |
+------------------------------------------+---------+---------+
REF PASS ratio: 0.356322
RES PASS ratio: 0.296703
***** ERROR: REF PASS ratio is abnormally low *****
***** ERROR: RES PASS ratio is abnormally low *****
***** ERROR: No REF execution test PASSed. Check execution engine configuration. *****
***** WARNING: No RES execution test FAILed. Check execution engine configuration. *****
o REGRESSIONS:
+------------------------------------------+---------+
| PASS now XPASS [PASS =>XPASS] | 2 |
| PASS now FAIL [PASS => FAIL] | 2 |
| PASS now KFAIL [PASS =>KFAIL] | 2 |
| PASS now UNRESOLVED [PASS =>UNRES] | 2 |
| ERROR appears [ =>ERROR] | 1 |
| XPASS now FAIL [XPASS=> FAIL] | 1 |
| XPASS now KFAIL [XPASS=>KFAIL] | 1 |
| XPASS appears [ =>XPASS] | 2 |
| XPASS now UNRESOLVED [XPASS=>UNRES] | 1 |
| FAIL appears [ => FAIL] | 2 |
| FAIL now UNRESOLVED [FAIL =>UNRES] | 1 |
| XFAIL now FAIL [XFAIL=> FAIL] | 1 |
| XFAIL now UNRESOLVED [XFAIL=>UNRES] | 1 |
| KFAIL appears [ =>KFAIL] | 2 |
| KFAIL now UNRESOLVED [KFAIL=>UNRES] | 1 |
| UNSUPPORTED now FAIL [UNSUP=> FAIL] | 1 |
| UNSUPPORTED now KFAIL [UNSUP=>KFAIL] | 1 |
| UNSUPPORTED now XPASS [UNSUP=>XPASS] | 1 |
| UNSUPP now UNRESOLVED [UNSUP=>UNRES] | 1 |
| UNTESTED now FAIL [UNTES=> FAIL] | 1 |
| UNTESTED now KFAIL [UNTES=>KFAIL] | 1 |
| UNTESTED now XPASS [UNTES=>XPASS] | 1 |
| UNTESTED now UNRESOLVED [UNTES=>UNRES] | 1 |
| UNRESOLVED now FAIL [UNRES=> FAIL] | 1 |
| UNRESOLVED now KFAIL [UNRES=>KFAIL] | 1 |
| UNRESOLVED now XPASS [UNRES=>XPASS] | 1 |
| UNRESOLVED appears [ =>UNRES] | 1 |
+------------------------------------------+---------+
| TOTAL_REGRESSIONS | 34 |
+------------------------------------------+---------+
- PASS now XPASS [PASS =>XPASS]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test2-hwdep-pass-xpass
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test2-pass-xpass
- PASS now FAIL [PASS => FAIL]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test3-hwdep-pass-fail
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test3-pass-fail
- PASS now KFAIL [PASS =>KFAIL]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test9-hwdep-pass-kfail
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test9-pass-kfail
- PASS now UNRESOLVED [PASS =>UNRES]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test8-hwdep-pass-unresolved
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test8-pass-unresolved
- ERROR appears [ =>ERROR]:
Executed from: gcc-compare-results:tests-9.exp
gcc-compare-results:tests-9.exp=(DejaGnu) proc "scan-dump-tree-not fail_test optimized 3" does not exist.
- XPASS now FAIL [XPASS=> FAIL]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test3-xpass-fail
- XPASS now KFAIL [XPASS=>KFAIL]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test9-xpass-kfail
- XPASS appears [ =>XPASS]:
Executed from: gcc-compare-results:tests-appear.exp
gcc-compare-results:tests-appear.exp=test2-xpass-appears
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test2-xpass-appears
- XPASS now UNRESOLVED [XPASS=>UNRES]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test8-xpass-unresolved
- FAIL appears [ => FAIL]:
Executed from: gcc-compare-results:tests-appear.exp
gcc-compare-results:tests-appear.exp=test3-fail-appears
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test3-fail-appears
- FAIL now UNRESOLVED [FAIL =>UNRES]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test8-fail-unresolved
- XFAIL now FAIL [XFAIL=> FAIL]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test3-xfail-fail
- XFAIL now UNRESOLVED [XFAIL=>UNRES]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test8-xfail-unresolved
- KFAIL appears [ =>KFAIL]:
Executed from: gcc-compare-results:tests-appear.exp
gcc-compare-results:tests-appear.exp=test5-kfail-appears
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test8-kfail-appears
- KFAIL now UNRESOLVED [KFAIL=>UNRES]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test8-kfail-unresolved
- UNSUPPORTED now FAIL [UNSUP=> FAIL]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test3-unsupported-fail
- UNSUPPORTED now KFAIL [UNSUP=>KFAIL]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test9-unsupported-kfail
- UNSUPPORTED now XPASS [UNSUP=>XPASS]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test2-unsupported-xpass
- UNSUPP now UNRESOLVED [UNSUP=>UNRES]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test7-unsupported-unresolved
- UNTESTED now FAIL [UNTES=> FAIL]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test3-untested-fail
- UNTESTED now KFAIL [UNTES=>KFAIL]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test9-untested-kfail
- UNTESTED now XPASS [UNTES=>XPASS]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test2-untested-xpass
- UNTESTED now UNRESOLVED [UNTES=>UNRES]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test7-untested-unresolved
- UNRESOLVED now FAIL [UNRES=> FAIL]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test3-unresolved-fail
- UNRESOLVED now KFAIL [UNRES=>KFAIL]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test9-unresolved-kfail
- UNRESOLVED now XPASS [UNRES=>XPASS]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test2-unresolved-xpass
- UNRESOLVED appears [ =>UNRES]:
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test7-unresolved-appears
o IMPROVEMENTS TO BE CHECKED:
+------------------------------------------+---------+
| PASS disappears [PASS => ] | 2 |
| PASS now XFAIL [PASS =>XFAIL] | 2 |
| New PASS [ => PASS] | 3 |
| PASS now UNSUPPORTED [PASS =>UNSUP] | 1 |
| PASS now UNTESTED [PASS =>UNTES] | 2 |
| XPASS now PASS [XPASS=> PASS] | 1 |
| XPASS now XFAIL [XPASS=>XFAIL] | 1 |
| XPASS now UNSUPPORTED [XPASS=>UNSUP] | 1 |
| XPASS now UNTESTED [XPASS=>UNTES] | 1 |
| XPASS disappears [XPASS=> ] | 1 |
| FAIL now PASS [FAIL => PASS] | 2 |
| FAIL now XPASS [FAIL =>XPASS] | 1 |
| FAIL now XFAIL [FAIL =>XFAIL] | 1 |
| FAIL now UNSUPPORTED [FAIL =>UNSUP] | 1 |
| FAIL now UNTESTED [FAIL =>UNTES] | 1 |
| FAIL disappears [FAIL => ] | 1 |
| XFAIL disappears [XFAIL=> ] | 1 |
| XFAIL now PASS [XFAIL=> PASS] | 1 |
| XFAIL now XPASS [XFAIL=>XPASS] | 1 |
| XFAIL now UNSUPPORTED [XFAIL=>UNSUP] | 1 |
| XFAIL now UNTESTED [XFAIL=>UNTES] | 1 |
| XFAIL appears [ =>XFAIL] | 2 |
| KFAIL now PASS [KFAIL=> PASS] | 1 |
| KFAIL now XPASS [KFAIL=>XPASS] | 1 |
| KFAIL now XFAIL [KFAIL=>XFAIL] | 1 |
| KFAIL now UNSUPPORTED [KFAIL=>UNSUP] | 1 |
| KFAIL now UNTESTED [KFAIL=>UNTES] | 1 |
| KFAIL disappears [KFAIL=> ] | 1 |
| UNSUPPORTED now PASS [UNSUP=> PASS] | 1 |
| UNSUPPORTED now XFAIL [UNSUP=>XFAIL] | 1 |
| UNSUPP now UNTESTED [UNSUP=>UNTES] | 1 |
| UNSUPPORTED disappears [UNSUP=> ] | 1 |
| UNSUPPORTED appears [ =>UNSUP] | 1 |
| UNTESTED now PASS [UNTES=> PASS] | 1 |
| UNTESTED now XFAIL [UNTES=>XFAIL] | 1 |
| UNTESTED now UNSUPP [UNTES=>UNSUP] | 1 |
| UNTESTED disappears [UNTES=> ] | 1 |
| UNTESTED appears [ =>UNTES] | 1 |
| UNRESOLVED now PASS [UNRES=> PASS] | 1 |
| UNRESOLVED now XFAIL [UNRES=>XFAIL] | 1 |
| UNRESOLVED now UNSUPP [UNRES=>UNSUP] | 1 |
| UNRESOLVED now UNTESTED [UNRES=>UNTES] | 1 |
| UNRESOLVED disappears [UNRES=> ] | 1 |
| ERROR disappears [ERROR=> ] | 1 |
| HW dependent cases [HW-DEPENDENT] | 2 |
+------------------------------------------+---------+
| TOTAL_IMPROVEMENTS_TO_BE_CHECKED | 51 |
+------------------------------------------+---------+
- PASS disappears [PASS => ]:
Executed from: gcc-compare-results:duplicate.exp
gcc-compare-results:duplicate.exp=test1-dup-pass-disappears
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test5-pass-disappears
- PASS now XFAIL [PASS =>XFAIL]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test4-hwdep-pass-xfail
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test4-pass-xfail
- New PASS [ => PASS]:
Executed from: gcc-compare-results:duplicate.exp
gcc-compare-results:duplicate.exp=test2-dup-pass-appears
Executed from: gcc-compare-results:tests-appear.exp
gcc-compare-results:tests-appear.exp=test1-pass-appears
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test1-pass-appears
- PASS now UNSUPPORTED [PASS =>UNSUP]:
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test6-pass-unsupported
- PASS now UNTESTED [PASS =>UNTES]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test7-hwdep-pass-untested
Executed from: gcc-compare-results:tests-1.exp
gcc-compare-results:tests-1.exp=test7-pass-untested
- XPASS now PASS [XPASS=> PASS]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test1-xpass-pass
- XPASS now XFAIL [XPASS=>XFAIL]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test4-xpass-xfail
- XPASS now UNSUPPORTED [XPASS=>UNSUP]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test6-xpass-unsupported
- XPASS now UNTESTED [XPASS=>UNTES]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test7-xpass-untested
- XPASS disappears [XPASS=> ]:
Executed from: gcc-compare-results:tests-2.exp
gcc-compare-results:tests-2.exp=test5-xpass-disappears
- FAIL now PASS [FAIL => PASS]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test1-fail-pass
Executed from: gcc-compare-results:tests-9.exp
gcc-compare-results:tests-9.exp=test1-fail-pass execution
- FAIL now XPASS [FAIL =>XPASS]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test2-fail-xpass
- FAIL now XFAIL [FAIL =>XFAIL]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test4-fail-xfail
- FAIL now UNSUPPORTED [FAIL =>UNSUP]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test6-fail-unsupported
- FAIL now UNTESTED [FAIL =>UNTES]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test7-fail-untested
- FAIL disappears [FAIL => ]:
Executed from: gcc-compare-results:tests-3.exp
gcc-compare-results:tests-3.exp=test5-fail-disappears
- XFAIL disappears [XFAIL=> ]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test5-xfail-disappears
- XFAIL now PASS [XFAIL=> PASS]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test1-xfail-pass
- XFAIL now XPASS [XFAIL=>XPASS]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test2-xfail-xpass
- XFAIL now UNSUPPORTED [XFAIL=>UNSUP]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test6-xfail-unsupported
- XFAIL now UNTESTED [XFAIL=>UNTES]:
Executed from: gcc-compare-results:tests-4.exp
gcc-compare-results:tests-4.exp=test7-xfail-untested
- XFAIL appears [ =>XFAIL]:
Executed from: gcc-compare-results:tests-appear.exp
gcc-compare-results:tests-appear.exp=test4-xfail-appears
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test4-xfail-appears
- KFAIL now PASS [KFAIL=> PASS]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test1-kfail-pass
- KFAIL now XPASS [KFAIL=>XPASS]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test2-kfail-xpass
- KFAIL now XFAIL [KFAIL=>XFAIL]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test4-kfail-xfail
- KFAIL now UNSUPPORTED [KFAIL=>UNSUP]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test6-kfail-unsupported
- KFAIL now UNTESTED [KFAIL=>UNTES]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test7-kfail-untested
- KFAIL disappears [KFAIL=> ]:
Executed from: gcc-compare-results:tests-10.exp
gcc-compare-results:tests-10.exp=test5-kfail-disappears
- UNSUPPORTED now PASS [UNSUP=> PASS]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test1-unsupported-pass
- UNSUPPORTED now XFAIL [UNSUP=>XFAIL]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test4-unsupported-xfail
- UNSUPP now UNTESTED [UNSUP=>UNTES]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test6-unsupported-untested
- UNSUPPORTED disappears [UNSUP=> ]:
Executed from: gcc-compare-results:tests-5.exp
gcc-compare-results:tests-5.exp=test5-unsupported-disappears
- UNSUPPORTED appears [ =>UNSUP]:
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test5-unsupported-appears
- UNTESTED now PASS [UNTES=> PASS]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test1-untested-pass
- UNTESTED now XFAIL [UNTES=>XFAIL]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test4-untested-xfail
- UNTESTED now UNSUPP [UNTES=>UNSUP]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test6-untested-unsupported
- UNTESTED disappears [UNTES=> ]:
Executed from: gcc-compare-results:tests-6.exp
gcc-compare-results:tests-6.exp=test5-untested-disappears
- UNTESTED appears [ =>UNTES]:
Executed from: gcc-compare-results:tests-appear2.exp
gcc-compare-results:tests-appear2.exp=test6-untested-appears
- UNRESOLVED now PASS [UNRES=> PASS]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test1-unresolved-pass
- UNRESOLVED now XFAIL [UNRES=>XFAIL]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test4-unresolved-xfail
- UNRESOLVED now UNSUPP [UNRES=>UNSUP]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test6-unresolved-unsupported
- UNRESOLVED now UNTESTED [UNRES=>UNTES]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test7-unresolved-untested
- UNRESOLVED disappears [UNRES=> ]:
Executed from: gcc-compare-results:tests-7.exp
gcc-compare-results:tests-7.exp=test5-unresolved-disappears
- ERROR disappears [ERROR=> ]:
Executed from: gcc-compare-results:tests-9.exp
gcc-compare-results:tests-9.exp=(DejaGnu) proc "scan-dump-tree-not fail_test optimized 1" does not exist.
- HW dependent cases [HW-DEPENDENT]:
Executed from: gcc-compare-results:hw-dep.exp
gcc-compare-results:hw-dep.exp=test5-hwdep-pass-disappears
gcc-compare-results:hw-dep.exp=test6-hwdep-pass-unsupported