Opened 10 years ago

Last modified 10 years ago

#47 new enhancement

Adding a test-run report to the test-suite.

Reported by: Dr. Ivan S. Zapreev Owned by:
Priority: minor Milestone: MRMC future
Component: Test Suite Version: 1.3.1
Keywords: Cc: nguen

Description (last modified by Dr. Ivan S. Zapreev)

I would suggest to add the accumulated survey (log) on the passed/failed tests at the end of the testing process. This log should also include all the .diff data for the failed tests.

It might be a god idea to have separate log files for internal, functional and performance tests:

=======INTERNAL TESTS=======

TESTS PASSED: ??
TESTS FAILED: ??

------------------
TESTS IN TOTAL: ??

-----FAILED TESTS DIFFS------

.out vs .golden

--- internal/bitset
<The diff file content goes here>
.....

The same kind of file should be present for the functional tests

For the performance tests, the format should be different, something like the following:

==========PERFORMANCE TESTS==========

--- <Full test case name>

TESTS PASSED: ??
TESTS FAILED: ??

--- <Full test case name>

TESTS PASSED: ??
TESTS FAILED: ??

.......
-----FAILED CASE TEST NAMES ------
<Full test case name>
<Full test case name>
<Full test case name>
.......

Note that for performance tests:

  1. The diff outputs are not necessary.
  2. The test case is repeted several times, so if there is at least one failure of this test case, it should be present in the list of failed test cases.

Change History (1)

comment:1 Changed 10 years ago by Dr. Ivan S. Zapreev

Description: modified (diff)

A minor lay out fix.

Note: See TracTickets for help on using tickets.