Manual Testing

How manual test results are handled in the log files and the test report document.

Some of the test cases in the test specification are not automated. For some, it is not possible to automate the test case. In others, automation will be added to DD-STAT in future releases.

These test cases must be run manually by the test engineer. This section describes how the automation interoperates with the engineer.

Detail.log file and report summary (HTML)

In the detail.log, manual test cases will be marked with [WARN] logging level. The warning will also show in the graphical report summary. Also, the End: statement will contain the text "manual".

[WARN] - Unknown: TFP711_013 is evaluated manually.
[INFO] - End: TFP711_013  manual  Unknown (Warning)

Test report document

In this Word document, each section of tests contains a table summarizing the pass/fail results. In the automated tests, the disposition column is set by the automation and is overwritten each time the automation runs. In the manual cases, this cell will not be overwritten. This allows the test engineer to manually type the test result into these cells, and the values are preserved from run to run. When you first open the report, these cells will be empty.

In the detailed section for each test case, the result will show as Unknown and the evaluation as Manual. Text which is written directly into the document under the boxed automation results will not be overwritten from run to run.