upstream/bind9
1
0
Fork 0
mirror of https://github.com/isc-projects/bind9.git synced 2026-06-10 23:19:59 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Detect missing system test results

Browse source 
At the end of each system test suite run, the system test framework
collects all existing test.output files from system test subdirectories
and produces bin/tests/system/systests.output from those files.
However, it does not check whether a test.output file was found for
every executed test.  Thus, if the test.output file is accidentally
deleted by the system test itself (e.g. due to an overly broad file
removal wildcard present in clean.sh), its output will not be included
in bin/tests/system/systests.output.  Since the result of each system
test suite run is determined by bin/tests/system/testsummary.sh, which
only operates on the contents of bin/tests/system/systests.output, this
can lead to test failures being ignored.  Fix by ensuring the number of
test results found in bin/tests/system/systests.output is equal to the
number of tests run and triggering a system test suite failure in case
of a discrepancy between these two values.
This commit is contained in:
Michał Kępień 2019-12-06 14:11:01 +01:00
parent 26ee43da1b
commit 3c3085be3c
1 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
bin/tests/system/testsummary.sh
View file

@ -56,4 +56,11 @@ if [ -n "${FAILED_TESTS}" ]; then
status=1
fi
RESULTS_FOUND=`grep -c 'R:[a-z0-9_-][a-z0-9_-]*:[A-Z][A-Z]*' systests.output`
TESTS_RUN=`echo "${SUBDIRS}" | wc -w`
if [ "${RESULTS_FOUND}" -ne "${TESTS_RUN}" ]; then
echofail "I:Found ${RESULTS_FOUND} test results, but ${TESTS_RUN} tests were run"
status=1
fi
exit $status