testsuite/expected-1-2.txt

Comparing:
REFERENCE:sum-1.txt
CURRENT:  sum-2.txt

                                             +---------+---------+
o  RUN STATUS:                               |   REF   |   RES   |
  +------------------------------------------+---------+---------+
  | Passes                            [PASS] |      22 |      16 |
  | Unexpected fails                  [FAIL] |      11 |      12 |
  | Errors                           [ERROR] |       1 |       1 |
  | Unexpected passes                [XPASS] |       9 |      11 |
  | Expected fails                   [XFAIL] |       9 |      11 |
  | Known fails                      [KFAIL] |       9 |      11 |
  | Unresolved                  [UNRESOLVED] |       9 |      10 |
  | Unsupported                [UNSUPPORTED] |       9 |      10 |
  | Untested                      [UNTESTED] |       9 |      10 |
  +------------------------------------------+---------+---------+

    REF PASS ratio: 0.356322
    RES PASS ratio: 0.296703
    ***** ERROR: REF PASS ratio is abnormally low *****
    ***** ERROR: RES PASS ratio is abnormally low *****
    ***** ERROR: No REF execution test PASSed. Check execution engine configuration. *****
    ***** WARNING: No RES execution test FAILed. Check execution engine configuration. *****

o  REGRESSIONS:
  +------------------------------------------+---------+
  | PASS now XPASS            [PASS =>XPASS] |       2 |
  | PASS now FAIL             [PASS => FAIL] |       2 |
  | PASS now KFAIL            [PASS =>KFAIL] |       2 |
  | PASS now UNRESOLVED       [PASS =>UNRES] |       2 |
  | ERROR appears             [     =>ERROR] |       1 |
  | XPASS now FAIL            [XPASS=> FAIL] |       1 |
  | XPASS now KFAIL           [XPASS=>KFAIL] |       1 |
  | XPASS appears             [     =>XPASS] |       2 |
  | XPASS now UNRESOLVED      [XPASS=>UNRES] |       1 |
  | FAIL appears              [     => FAIL] |       2 |
  | FAIL now UNRESOLVED       [FAIL =>UNRES] |       1 |
  | XFAIL now FAIL            [XFAIL=> FAIL] |       1 |
  | XFAIL now UNRESOLVED      [XFAIL=>UNRES] |       1 |
  | KFAIL appears             [     =>KFAIL] |       2 |
  | KFAIL now UNRESOLVED      [KFAIL=>UNRES] |       1 |
  | UNSUPPORTED now FAIL      [UNSUP=> FAIL] |       1 |
  | UNSUPPORTED now KFAIL     [UNSUP=>KFAIL] |       1 |
  | UNSUPPORTED now XPASS     [UNSUP=>XPASS] |       1 |
  | UNSUPP now UNRESOLVED     [UNSUP=>UNRES] |       1 |
  | UNTESTED now FAIL         [UNTES=> FAIL] |       1 |
  | UNTESTED now KFAIL        [UNTES=>KFAIL] |       1 |
  | UNTESTED now XPASS        [UNTES=>XPASS] |       1 |
  | UNTESTED now UNRESOLVED   [UNTES=>UNRES] |       1 |
  | UNRESOLVED now FAIL       [UNRES=> FAIL] |       1 |
  | UNRESOLVED now KFAIL      [UNRES=>KFAIL] |       1 |
  | UNRESOLVED now XPASS      [UNRES=>XPASS] |       1 |
  | UNRESOLVED appears        [     =>UNRES] |       1 |
  +------------------------------------------+---------+
  | TOTAL_REGRESSIONS                        |      34 |
  +------------------------------------------+---------+


o  IMPROVEMENTS TO BE CHECKED:
  +------------------------------------------+---------+
  | PASS disappears           [PASS =>     ] |       2 |
  | PASS now XFAIL            [PASS =>XFAIL] |       2 |
  | New PASS                  [     => PASS] |       3 |
  | PASS now UNSUPPORTED      [PASS =>UNSUP] |       1 |
  | PASS now UNTESTED         [PASS =>UNTES] |       2 |
  | XPASS now PASS            [XPASS=> PASS] |       1 |
  | XPASS now XFAIL           [XPASS=>XFAIL] |       1 |
  | XPASS now UNSUPPORTED     [XPASS=>UNSUP] |       1 |
  | XPASS now UNTESTED        [XPASS=>UNTES] |       1 |
  | XPASS disappears          [XPASS=>     ] |       1 |
  | FAIL now PASS             [FAIL => PASS] |       2 |
  | FAIL now XPASS            [FAIL =>XPASS] |       1 |
  | FAIL now XFAIL            [FAIL =>XFAIL] |       1 |
  | FAIL now UNSUPPORTED      [FAIL =>UNSUP] |       1 |
  | FAIL now UNTESTED         [FAIL =>UNTES] |       1 |
  | FAIL disappears           [FAIL =>     ] |       1 |
  | XFAIL disappears          [XFAIL=>     ] |       1 |
  | XFAIL now PASS            [XFAIL=> PASS] |       1 |
  | XFAIL now XPASS           [XFAIL=>XPASS] |       1 |
  | XFAIL now UNSUPPORTED     [XFAIL=>UNSUP] |       1 |
  | XFAIL now UNTESTED        [XFAIL=>UNTES] |       1 |
  | XFAIL appears             [     =>XFAIL] |       2 |
  | KFAIL now PASS            [KFAIL=> PASS] |       1 |
  | KFAIL now XPASS           [KFAIL=>XPASS] |       1 |
  | KFAIL now XFAIL           [KFAIL=>XFAIL] |       1 |
  | KFAIL now UNSUPPORTED     [KFAIL=>UNSUP] |       1 |
  | KFAIL now UNTESTED        [KFAIL=>UNTES] |       1 |
  | KFAIL disappears          [KFAIL=>     ] |       1 |
  | UNSUPPORTED now PASS      [UNSUP=> PASS] |       1 |
  | UNSUPPORTED now XFAIL     [UNSUP=>XFAIL] |       1 |
  | UNSUPP now UNTESTED       [UNSUP=>UNTES] |       1 |
  | UNSUPPORTED disappears    [UNSUP=>     ] |       1 |
  | UNSUPPORTED appears       [     =>UNSUP] |       1 |
  | UNTESTED now PASS         [UNTES=> PASS] |       1 |
  | UNTESTED now XFAIL        [UNTES=>XFAIL] |       1 |
  | UNTESTED now UNSUPP       [UNTES=>UNSUP] |       1 |
  | UNTESTED disappears       [UNTES=>     ] |       1 |
  | UNTESTED appears          [     =>UNTES] |       1 |
  | UNRESOLVED now PASS       [UNRES=> PASS] |       1 |
  | UNRESOLVED now XFAIL      [UNRES=>XFAIL] |       1 |
  | UNRESOLVED now UNSUPP     [UNRES=>UNSUP] |       1 |
  | UNRESOLVED now UNTESTED   [UNRES=>UNTES] |       1 |
  | UNRESOLVED disappears     [UNRES=>     ] |       1 |
  | ERROR disappears          [ERROR=>     ] |       1 |
  | HW dependent cases        [HW-DEPENDENT] |       2 |
  +------------------------------------------+---------+
  | TOTAL_IMPROVEMENTS_TO_BE_CHECKED         |      51 |
  +------------------------------------------+---------+