| Comparing: |
| REFERENCE:sum-1.txt |
| CURRENT: sum-2.txt |
| |
| ` +---------+---------+ |
| o RUN STATUS: | REF | RES | |
| +------------------------------------------+---------+---------+ |
| | Passes [PASS] | 17 | 12 | |
| | Unexpected fails [FAIL] | 10 | 11 | |
| | Errors [ERROR] | 1 | 1 | |
| | Unexpected passes [XPASS] | 8 | 10 | |
| | Expected fails [XFAIL] | 8 | 10 | |
| | Unresolved [UNRESOLVED] | 7 | 8 | |
| | Unsupported [UNSUPPORTED] | 7 | 8 | |
| | Untested [UNTESTED] | 7 | 8 | |
| +------------------------------------------+---------+---------+ |
| |
| REF PASS ratio: 0.390625 |
| RES PASS ratio: 0.328358 |
| ***** ERROR: REF PASS ratio is abnormally low ***** |
| ***** ERROR: RES PASS ratio is abnormally low ***** |
| ***** ERROR: No REF execution test PASSed. Check execution engine configuration. ***** |
| ***** WARNING: No RES execution test FAILed. Check execution engine configuration. ***** |
| |
| o REGRESSIONS: |
| +------------------------------------------+---------+ |
| | PASS now XPASS [PASS =>XPASS] | 2 | |
| | PASS now FAIL [PASS => FAIL] | 2 | |
| | PASS now UNRESOLVED [PASS =>UNRES] | 2 | |
| | ERROR appears [ =>ERROR] | 1 | |
| | XPASS now FAIL [XPASS=> FAIL] | 1 | |
| | XPASS appears [ =>XPASS] | 1 | |
| | XPASS now UNRESOLVED [XPASS=>UNRES] | 1 | |
| | FAIL appears [ => FAIL] | 1 | |
| | FAIL now UNRESOLVED [FAIL =>UNRES] | 1 | |
| | XFAIL now FAIL [XFAIL=> FAIL] | 1 | |
| | XFAIL now UNRESOLVED [XFAIL=>UNRES] | 1 | |
| | UNSUPPORTED now FAIL [UNSUP=> FAIL] | 1 | |
| | UNSUPPORTED now XPASS [UNSUP=>XPASS] | 1 | |
| | UNSUPP now UNRESOLVED [UNSUP=>UNRES] | 1 | |
| | UNTESTED now FAIL [UNTES=> FAIL] | 1 | |
| | UNTESTED now XPASS [UNTES=>XPASS] | 1 | |
| | UNTESTED now UNRESOLVED [UNTES=>UNRES] | 1 | |
| | UNRESOLVED now FAIL [UNRES=> FAIL] | 1 | |
| | UNRESOLVED now XPASS [UNRES=>XPASS] | 1 | |
| | UNRESOLVED appears [ =>UNRES] | 1 | |
| +------------------------------------------+---------+ |
| | TOTAL_REGRESSIONS | 23 | |
| +------------------------------------------+---------+ |
| |
| - PASS now XPASS [PASS =>XPASS]: |
| |
| Executed from: hw-dep.exp |
| test2-hwdep-pass-xpass |
| Executed from: tests-1.exp |
| test2-pass-xpass |
| |
| |
| - PASS now FAIL [PASS => FAIL]: |
| |
| Executed from: hw-dep.exp |
| test3-hwdep-pass-fail |
| Executed from: tests-1.exp |
| test3-pass-fail |
| |
| |
| - PASS now UNRESOLVED [PASS =>UNRES]: |
| |
| Executed from: hw-dep.exp |
| test8-hwdep-pass-unresolved |
| Executed from: tests-1.exp |
| test8-pass-unresolved |
| |
| |
| - ERROR appears [ =>ERROR]: |
| |
| Executed from: tests-9.exp |
| (DejaGnu) proc "scan-dump-tree-not fail_test optimized 3" does not exist. |
| |
| |
| - XPASS now FAIL [XPASS=> FAIL]: |
| |
| Executed from: test-2.exp |
| test3-xpass-fail |
| |
| |
| - XPASS appears [ =>XPASS]: |
| |
| Executed from: tests-appear2.exp |
| test2-xpass-appears |
| |
| |
| - XPASS now UNRESOLVED [XPASS=>UNRES]: |
| |
| Executed from: test-2.exp |
| test8-xpass-unresolved |
| |
| |
| - FAIL appears [ => FAIL]: |
| |
| Executed from: tests-appear2.exp |
| test3-fail-appears |
| |
| |
| - FAIL now UNRESOLVED [FAIL =>UNRES]: |
| |
| Executed from: tests-3.exp |
| test8-fail-unresolved |
| |
| |
| - XFAIL now FAIL [XFAIL=> FAIL]: |
| |
| Executed from: tests-4.exp |
| test3-xfail-fail |
| |
| |
| - XFAIL now UNRESOLVED [XFAIL=>UNRES]: |
| |
| Executed from: tests-4.exp |
| test8-xfail-unresolved |
| |
| |
| - UNSUPPORTED now FAIL [UNSUP=> FAIL]: |
| |
| Executed from: tests-5.exp |
| test3-unsupported-fail |
| |
| |
| - UNSUPPORTED now XPASS [UNSUP=>XPASS]: |
| |
| Executed from: tests-5.exp |
| test2-unsupported-xpass |
| |
| |
| - UNSUPP now UNRESOLVED [UNSUP=>UNRES]: |
| |
| Executed from: tests-5.exp |
| test7-unsupported-unresolved |
| |
| |
| - UNTESTED now FAIL [UNTES=> FAIL]: |
| |
| Executed from: tests-6.exp |
| test3-untested-fail |
| |
| |
| - UNTESTED now XPASS [UNTES=>XPASS]: |
| |
| Executed from: tests-6.exp |
| test2-untested-xpass |
| |
| |
| - UNTESTED now UNRESOLVED [UNTES=>UNRES]: |
| |
| Executed from: tests-6.exp |
| test7-untested-unresolved |
| |
| |
| - UNRESOLVED now FAIL [UNRES=> FAIL]: |
| |
| Executed from: tests-7.exp |
| test3-unresolved-fail |
| |
| |
| - UNRESOLVED now XPASS [UNRES=>XPASS]: |
| |
| Executed from: tests-7.exp |
| test2-unresolved-xpass |
| |
| |
| - UNRESOLVED appears [ =>UNRES]: |
| |
| Executed from: tests-appear2.exp |
| test4-unresolved-appears |
| |
| |
| |
| o IMPROVEMENTS TO BE CHECKED: |
| +------------------------------------------+---------+ |
| | PASS disappears [PASS => ] | 1 | |
| | PASS now XFAIL [PASS =>XFAIL] | 2 | |
| | New PASS [ => PASS] | 1 | |
| | PASS now UNSUPPORTED [PASS =>UNSUP] | 1 | |
| | PASS now UNTESTED [PASS =>UNTES] | 2 | |
| | XPASS now PASS [XPASS=> PASS] | 1 | |
| | XPASS now XFAIL [XPASS=>XFAIL] | 1 | |
| | XPASS now UNSUPPORTED [XPASS=>UNSUP] | 1 | |
| | XPASS now UNTESTED [XPASS=>UNTES] | 1 | |
| | XPASS disappears [XPASS=> ] | 1 | |
| | FAIL now PASS [FAIL => PASS] | 2 | |
| | FAIL now XPASS [FAIL =>XPASS] | 1 | |
| | FAIL now XFAIL [FAIL =>XFAIL] | 1 | |
| | FAIL now UNSUPPORTED [FAIL =>UNSUP] | 1 | |
| | FAIL now UNTESTED [FAIL =>UNTES] | 1 | |
| | FAIL disappears [FAIL => ] | 1 | |
| | XFAIL disappears [XFAIL=> ] | 1 | |
| | XFAIL now PASS [XFAIL=> PASS] | 1 | |
| | XFAIL now XPASS [XFAIL=>XPASS] | 1 | |
| | XFAIL now UNSUPPORTED [XFAIL=>UNSUP] | 1 | |
| | XFAIL now UNTESTED [XFAIL=>UNTES] | 1 | |
| | XFAIL appears [ =>XFAIL] | 1 | |
| | UNSUPPORTED now PASS [UNSUP=> PASS] | 1 | |
| | UNSUPPORTED now XFAIL [UNSUP=>XFAIL] | 1 | |
| | UNSUPP now UNTESTED [UNSUP=>UNTES] | 1 | |
| | UNSUPPORTED disappears [UNSUP=> ] | 1 | |
| | UNSUPPORTED appears [ =>UNSUP] | 1 | |
| | UNTESTED now PASS [UNTES=> PASS] | 1 | |
| | UNTESTED now XFAIL [UNTES=>XFAIL] | 1 | |
| | UNTESTED now UNSUPP [UNTES=>UNSUP] | 1 | |
| | UNTESTED disappears [UNTES=> ] | 1 | |
| | UNTESTED appears [ =>UNTES] | 1 | |
| | UNRESOLVED now PASS [UNRES=> PASS] | 1 | |
| | UNRESOLVED now XFAIL [UNRES=>XFAIL] | 1 | |
| | UNRESOLVED now UNSUPP [UNRES=>UNSUP] | 1 | |
| | UNRESOLVED now UNTESTED [UNRES=>UNTES] | 1 | |
| | UNRESOLVED disappears [UNRES=> ] | 1 | |
| | ERROR disappears [ERROR=> ] | 1 | |
| | HW dependent cases [HW-DEPENDENT] | 2 | |
| +------------------------------------------+---------+ |
| | TOTAL_IMPROVEMENTS_TO_BE_CHECKED | 41 | |
| +------------------------------------------+---------+ |
| |
| - PASS disappears [PASS => ]: |
| |
| Executed from: tests-1.exp |
| test5-pass-disappears |
| |
| |
| - PASS now XFAIL [PASS =>XFAIL]: |
| |
| Executed from: hw-dep.exp |
| test4-hwdep-pass-xfail |
| Executed from: tests-1.exp |
| test4-pass-xfail |
| |
| |
| - New PASS [ => PASS]: |
| |
| Executed from: tests-appear2.exp |
| test1-pass-appears |
| |
| |
| - PASS now UNSUPPORTED [PASS =>UNSUP]: |
| |
| Executed from: tests-1.exp |
| test6-pass-unsupported |
| |
| |
| - PASS now UNTESTED [PASS =>UNTES]: |
| |
| Executed from: hw-dep.exp |
| test7-hwdep-pass-untested |
| Executed from: tests-1.exp |
| test7-pass-untested |
| |
| |
| - XPASS now PASS [XPASS=> PASS]: |
| |
| Executed from: test-2.exp |
| test1-xpass-pass |
| |
| |
| - XPASS now XFAIL [XPASS=>XFAIL]: |
| |
| Executed from: test-2.exp |
| test4-xpass-xfail |
| |
| |
| - XPASS now UNSUPPORTED [XPASS=>UNSUP]: |
| |
| Executed from: test-2.exp |
| test6-xpass-unsupported |
| |
| |
| - XPASS now UNTESTED [XPASS=>UNTES]: |
| |
| Executed from: test-2.exp |
| test7-xpass-untested |
| |
| |
| - XPASS disappears [XPASS=> ]: |
| |
| Executed from: tests-2.exp |
| test5-xpass-disappears |
| |
| |
| - FAIL now PASS [FAIL => PASS]: |
| |
| Executed from: tests-3.exp |
| test1-fail-pass |
| Executed from: tests-9.exp |
| test1-fail-pass execution |
| |
| |
| - FAIL now XPASS [FAIL =>XPASS]: |
| |
| Executed from: tests-3.exp |
| test2-fail-xpass |
| |
| |
| - FAIL now XFAIL [FAIL =>XFAIL]: |
| |
| Executed from: tests-3.exp |
| test4-fail-xfail |
| |
| |
| - FAIL now UNSUPPORTED [FAIL =>UNSUP]: |
| |
| Executed from: tests-3.exp |
| test6-fail-unsupported |
| |
| |
| - FAIL now UNTESTED [FAIL =>UNTES]: |
| |
| Executed from: tests-3.exp |
| test7-fail-untested |
| |
| |
| - FAIL disappears [FAIL => ]: |
| |
| Executed from: tests-3.exp |
| test5-fail-disappears |
| |
| |
| - XFAIL disappears [XFAIL=> ]: |
| |
| Executed from: tests-4.exp |
| test5-xfail-disappears |
| |
| |
| - XFAIL now PASS [XFAIL=> PASS]: |
| |
| Executed from: tests-4.exp |
| test1-xfail-pass |
| |
| |
| - XFAIL now XPASS [XFAIL=>XPASS]: |
| |
| Executed from: tests-4.exp |
| test2-xfail-xpass |
| |
| |
| - XFAIL now UNSUPPORTED [XFAIL=>UNSUP]: |
| |
| Executed from: tests-4.exp |
| test6-xfail-unsupported |
| |
| |
| - XFAIL now UNTESTED [XFAIL=>UNTES]: |
| |
| Executed from: tests-4.exp |
| test7-xfail-untested |
| |
| |
| - XFAIL appears [ =>XFAIL]: |
| |
| Executed from: tests-appear2.exp |
| test4-xfail-appears |
| |
| |
| - UNSUPPORTED now PASS [UNSUP=> PASS]: |
| |
| Executed from: tests-5.exp |
| test1-unsupported-pass |
| |
| |
| - UNSUPPORTED now XFAIL [UNSUP=>XFAIL]: |
| |
| Executed from: tests-5.exp |
| test4-unsupported-xfail |
| |
| |
| - UNSUPP now UNTESTED [UNSUP=>UNTES]: |
| |
| Executed from: tests-5.exp |
| test6-unsupported-untested |
| |
| |
| - UNSUPPORTED disappears [UNSUP=> ]: |
| |
| Executed from: tests-5.exp |
| test5-unsupported-disappears |
| |
| |
| - UNSUPPORTED appears [ =>UNSUP]: |
| |
| Executed from: tests-appear2.exp |
| test4-unsupported-appears |
| |
| |
| - UNTESTED now PASS [UNTES=> PASS]: |
| |
| Executed from: tests-6.exp |
| test1-untested-pass |
| |
| |
| - UNTESTED now XFAIL [UNTES=>XFAIL]: |
| |
| Executed from: tests-6.exp |
| test4-untested-xfail |
| |
| |
| - UNTESTED now UNSUPP [UNTES=>UNSUP]: |
| |
| Executed from: tests-6.exp |
| test6-untested-unsupported |
| |
| |
| - UNTESTED disappears [UNTES=> ]: |
| |
| Executed from: tests-6.exp |
| test5-untested-disappears |
| |
| |
| - UNTESTED appears [ =>UNTES]: |
| |
| Executed from: tests-appear2.exp |
| test4-untested-appears |
| |
| |
| - UNRESOLVED now PASS [UNRES=> PASS]: |
| |
| Executed from: tests-7.exp |
| test1-unresolved-pass |
| |
| |
| - UNRESOLVED now XFAIL [UNRES=>XFAIL]: |
| |
| Executed from: tests-7.exp |
| test4-unresolved-xfail |
| |
| |
| - UNRESOLVED now UNSUPP [UNRES=>UNSUP]: |
| |
| Executed from: tests-7.exp |
| test6-unresolved-unsupported |
| |
| |
| - UNRESOLVED now UNTESTED [UNRES=>UNTES]: |
| |
| Executed from: tests-7.exp |
| test7-unresolved-untested |
| |
| |
| - UNRESOLVED disappears [UNRES=> ]: |
| |
| Executed from: tests-7.exp |
| test5-unresolved-disappears |
| |
| |
| - ERROR disappears [ERROR=> ]: |
| |
| Executed from: tests-9.exp |
| (DejaGnu) proc "scan-dump-tree-not fail_test optimized 1" does not exist. |
| |
| |
| - HW dependent cases [HW-DEPENDENT]: |
| |
| Executed from: hw-dep.exp |
| test5-hwdep-pass-disappears |
| test6-hwdep-pass-unsupported |
| |
| |
| |