Opened 11 years ago
Last modified 11 years ago
#47 new enhancement
Adding a test-run report to the test-suite.
Reported by: | Dr. Ivan S. Zapreev | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | MRMC future |
Component: | Test Suite | Version: | 1.3.1 |
Keywords: | Cc: | nguen |
Description (last modified by )
I would suggest to add the accumulated survey (log) on the passed/failed tests at the end of the testing process. This log should also include all the .diff data for the failed tests.
It might be a god idea to have separate log files for internal, functional and performance tests:
=======INTERNAL TESTS=======
TESTS PASSED: ??
TESTS FAILED: ??
------------------
TESTS IN TOTAL: ??
-----FAILED TESTS DIFFS------
.out vs .golden
--- internal/bitset
<The diff file content goes here>
.....
The same kind of file should be present for the functional tests
For the performance tests, the format should be different, something like the following:
==========PERFORMANCE TESTS==========
--- <Full test case name>
TESTS PASSED: ??
TESTS FAILED: ??
--- <Full test case name>
TESTS PASSED: ??
TESTS FAILED: ??
.......
-----FAILED CASE TEST NAMES ------
<Full test case name>
<Full test case name>
<Full test case name>
.......
Note that for performance tests:
- The diff outputs are not necessary.
- The test case is repeted several times, so if there is at least one failure of this test case, it should be present in the list of failed test cases.
A minor lay out fix.