diff --git a/bin/tests/system/README b/bin/tests/system/README index 14f99088dc..be42bea54e 100644 --- a/bin/tests/system/README +++ b/bin/tests/system/README @@ -284,24 +284,8 @@ allowing the output that caused the problem message to be identified. Re-Running the Tests --- If there is a requirement to re-run a test (or the entire test suite), the -files produced by the tests should be deleted first. Normally, these files are -deleted if the test succeeds but are retained on error. The legacy.run.sh -script automatically calls a given test's clean.sh script before invoking its -setup.sh script. - -Deletion of the files produced by the set of tests (e.g. after the execution -of "runall.sh") can be carried out using the command: - - sh cleanall.sh - -or - - make testclean - -(Note that the Makefile has two other targets for cleaning up files: "clean" -will delete all the files produced by the tests, as well as the object and -executable files used by the tests. "distclean" does all the work of "clean" -as well as deleting configuration files produced by "configure".) +files produced by the tests should be deleted first. This is no longer +supported by the legacy runner and the pytest runner should be used instead. Developer Notes @@ -337,11 +321,6 @@ tests.sh Runs the actual tests. This file is mandatory. tests_sh_xyz.py A glue file for the pytest runner for executing shell tests. -clean.sh Run at the end to clean up temporary files, but only if the test - was completed successfully and its running was not inhibited by the - "-n" switch being passed to "legacy.run.sh". Otherwise the - temporary files are left in place for inspection. - ns These subdirectories contain test name servers that can be queried or can interact with each other. The value of N indicates the address the server listens on: for example, ns2 listens on @@ -424,7 +403,6 @@ arguments, e.g.: (cd mytest ; sh prereq.sh -D xyz) (cd mytest ; sh setup.sh -D xyz) (cd mytest ; sh tests.sh -D xyz) - (cd mytest ; sh clean.sh -D xyz) No arguments will be passed to the test scripts if the test is run as part of a run of the full test suite (e.g. the tests are started with "runall.sh"). @@ -567,12 +545,6 @@ following contents. def test_xyz(run_tests_sh): run_tests_sh() -clean.sh ---- -The inverse of "setup.sh", this is invoked by the framework to clean up the -test directory. It should delete all files that have been created by the test -during its run. - Starting Nameservers --- @@ -854,15 +826,4 @@ abnormally, e.g. core files, PID files etc., are stored in the test directory. test is written to bin/tests/system/. This file is only produced when the test is run as part of the entire test suite (e.g. via "runall.sh"). -If the test fails, all these files are retained. But if the test succeeds, -they are cleaned up at different times: - -1. Files generated by the test itself are cleaned up by the test's own -"clean.sh", which is called from "legacy.run.sh". - -2. Files that may not be cleaned up if named exits abnormally can be removed -using the "cleanall.sh" script. - -3. "test.output.*" files are deleted when the test suite ends. At this point, -the file "testsummary.sh" is called which concatenates all the "test.output.*" -files into a single "systests.output" file before deleting them. +If the test fails, all these files are retained. diff --git a/bin/tests/system/acl/setup.sh b/bin/tests/system/acl/setup.sh index 7dcd87244b..f563d78cdd 100644 --- a/bin/tests/system/acl/setup.sh +++ b/bin/tests/system/acl/setup.sh @@ -13,7 +13,6 @@ . ../conf.sh -$SHELL clean.sh $SHELL ${TOP_SRCDIR}/bin/tests/system/genzone.sh 2 3 >ns2/example.db $SHELL ${TOP_SRCDIR}/bin/tests/system/genzone.sh 2 3 >ns2/tsigzone.db copy_setports ns2/named1.conf.in ns2/named.conf diff --git a/bin/tests/system/additional/setup.sh b/bin/tests/system/additional/setup.sh index 2cbc08d7e9..d117b4c062 100644 --- a/bin/tests/system/additional/setup.sh +++ b/bin/tests/system/additional/setup.sh @@ -13,7 +13,6 @@ . ../conf.sh -$SHELL clean.sh copy_setports ns1/named1.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/catz/setup.sh b/bin/tests/system/catz/setup.sh index eb2e53bd1a..c33ce7f1c1 100644 --- a/bin/tests/system/catz/setup.sh +++ b/bin/tests/system/catz/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named1.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/checkds/setup.sh b/bin/tests/system/checkds/setup.sh index 848f0c822f..0fc415e997 100644 --- a/bin/tests/system/checkds/setup.sh +++ b/bin/tests/system/checkds/setup.sh @@ -16,8 +16,6 @@ set -e -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/checknames/setup.sh b/bin/tests/system/checknames/setup.sh index d80e746223..25f05ef33f 100644 --- a/bin/tests/system/checknames/setup.sh +++ b/bin/tests/system/checknames/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/conf.sh.common b/bin/tests/system/conf.sh.common index c491a6926e..9022e4a418 100644 --- a/bin/tests/system/conf.sh.common +++ b/bin/tests/system/conf.sh.common @@ -432,8 +432,7 @@ private_type_record() { # has been appended since the last time we read it. # # Calling some of these functions causes temporary *.prev files to be -# created that need to be cleaned up manually (usually by a given system -# test's clean.sh script). +# created. # # Note that unlike other nextpart*() functions, nextpartread() is not # meant to be directly used in system tests; its sole purpose is to diff --git a/bin/tests/system/conftest.py b/bin/tests/system/conftest.py index e24e2d3b2f..2f48648fbf 100644 --- a/bin/tests/system/conftest.py +++ b/bin/tests/system/conftest.py @@ -375,8 +375,6 @@ def system_test_dir(request, env, system_test_name, expected_artifacts): This fixture is responsible for creating (and potentially removing) a copy of the system test directory which is used as a temporary directory for the test execution. - - FUTURE: This removes the need to have clean.sh scripts. """ def get_test_result(): diff --git a/bin/tests/system/cpu/setup.sh b/bin/tests/system/cpu/setup.sh index 9676770adb..7c9de7a6e7 100644 --- a/bin/tests/system/cpu/setup.sh +++ b/bin/tests/system/cpu/setup.sh @@ -16,6 +16,4 @@ set -e -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/dnssec/setup.sh b/bin/tests/system/dnssec/setup.sh index db0913d369..dc53ea83e2 100644 --- a/bin/tests/system/dnssec/setup.sh +++ b/bin/tests/system/dnssec/setup.sh @@ -16,8 +16,6 @@ set -e -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/dupsigs/setup.sh b/bin/tests/system/dupsigs/setup.sh index 563de434d2..4e45a8ea1c 100644 --- a/bin/tests/system/dupsigs/setup.sh +++ b/bin/tests/system/dupsigs/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - test -r $RANDFILE || $GENRANDOM 800 $RANDFILE copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/emptyzones/setup.sh b/bin/tests/system/emptyzones/setup.sh index 935067fb45..f3f0088a23 100644 --- a/bin/tests/system/emptyzones/setup.sh +++ b/bin/tests/system/emptyzones/setup.sh @@ -13,5 +13,4 @@ . ../conf.sh -$SHELL clean.sh copy_setports ns1/named1.conf.in ns1/named.conf diff --git a/bin/tests/system/forward/setup.sh b/bin/tests/system/forward/setup.sh index 33d33ccc9a..5715e9ad09 100644 --- a/bin/tests/system/forward/setup.sh +++ b/bin/tests/system/forward/setup.sh @@ -13,7 +13,6 @@ . ../conf.sh -$SHELL clean.sh copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named1.conf.in ns3/named.conf diff --git a/bin/tests/system/geoip2/setup.sh b/bin/tests/system/geoip2/setup.sh index a0eaaf713b..b9ec49f29d 100644 --- a/bin/tests/system/geoip2/setup.sh +++ b/bin/tests/system/geoip2/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns2/named1.conf.in ns2/named.conf for i in 1 2 3 4 5 6 7 other bogus; do diff --git a/bin/tests/system/idna/setup.sh b/bin/tests/system/idna/setup.sh index 2ac2ce8f0d..82240a7c1b 100644 --- a/bin/tests/system/idna/setup.sh +++ b/bin/tests/system/idna/setup.sh @@ -13,5 +13,4 @@ . ../conf.sh -$SHELL clean.sh copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/include-multiplecfg/setup.sh b/bin/tests/system/include-multiplecfg/setup.sh index 643e872ba1..3638ba5e5a 100644 --- a/bin/tests/system/include-multiplecfg/setup.sh +++ b/bin/tests/system/include-multiplecfg/setup.sh @@ -13,6 +13,4 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns2/named.conf.in ns2/named.conf diff --git a/bin/tests/system/ixfr/setup.sh b/bin/tests/system/ixfr/setup.sh index 0c5a2829f2..5b544411b0 100644 --- a/bin/tests/system/ixfr/setup.sh +++ b/bin/tests/system/ixfr/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns3/named.conf.in ns3/named.conf copy_setports ns4/named.conf.in ns4/named.conf diff --git a/bin/tests/system/journal/setup.sh b/bin/tests/system/journal/setup.sh index 6017f1c3b2..e50cd82f03 100644 --- a/bin/tests/system/journal/setup.sh +++ b/bin/tests/system/journal/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf cp ns1/generic.db.in ns1/changed.db cp ns1/changed.ver1.jnl.saved ns1/changed.db.jnl diff --git a/bin/tests/system/kasp/setup.sh b/bin/tests/system/kasp/setup.sh index 1d11ba96fd..0922afa944 100644 --- a/bin/tests/system/kasp/setup.sh +++ b/bin/tests/system/kasp/setup.sh @@ -16,8 +16,6 @@ set -e -$SHELL clean.sh - mkdir keys mkdir ns3/keys diff --git a/bin/tests/system/legacy/build.sh b/bin/tests/system/legacy/build.sh index fcde4e1036..2460fd784e 100644 --- a/bin/tests/system/legacy/build.sh +++ b/bin/tests/system/legacy/build.sh @@ -13,9 +13,5 @@ . ../conf.sh -$SHELL clean.sh - (cd ns6 && $SHELL -e sign.sh) (cd ns7 && $SHELL -e sign.sh) - -$SHELL clean.sh diff --git a/bin/tests/system/logfileconfig/setup.sh b/bin/tests/system/logfileconfig/setup.sh index 7ad9bb0bbd..8884af7dc7 100644 --- a/bin/tests/system/logfileconfig/setup.sh +++ b/bin/tests/system/logfileconfig/setup.sh @@ -13,6 +13,4 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.plain.in ns1/named.conf diff --git a/bin/tests/system/masterfile/setup.sh b/bin/tests/system/masterfile/setup.sh index 46c18c406a..6929ec541d 100644 --- a/bin/tests/system/masterfile/setup.sh +++ b/bin/tests/system/masterfile/setup.sh @@ -13,6 +13,5 @@ . ../conf.sh -$SHELL clean.sh copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf diff --git a/bin/tests/system/masterformat/setup.sh b/bin/tests/system/masterformat/setup.sh index f5d52cf23b..99556f5002 100755 --- a/bin/tests/system/masterformat/setup.sh +++ b/bin/tests/system/masterformat/setup.sh @@ -14,8 +14,6 @@ # shellcheck source=conf.sh . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/mirror/setup.sh b/bin/tests/system/mirror/setup.sh index 21d10931f4..134c4ad0c1 100644 --- a/bin/tests/system/mirror/setup.sh +++ b/bin/tests/system/mirror/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/nsec3/setup.sh b/bin/tests/system/nsec3/setup.sh index 3019361f08..41b0bb2075 100644 --- a/bin/tests/system/nsec3/setup.sh +++ b/bin/tests/system/nsec3/setup.sh @@ -16,8 +16,6 @@ set -e -$SHELL clean.sh - copy_setports ns2/named.conf.in ns2/named.conf ( cd ns2 diff --git a/bin/tests/system/nslookup/setup.sh b/bin/tests/system/nslookup/setup.sh index 67d3ccf642..5989f880c9 100644 --- a/bin/tests/system/nslookup/setup.sh +++ b/bin/tests/system/nslookup/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - $SHELL ${TOP_SRCDIR}/bin/tests/system/genzone.sh 1 >ns1/example.db copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/nsupdate/setup.sh b/bin/tests/system/nsupdate/setup.sh index e901ea05b8..4ee045facc 100644 --- a/bin/tests/system/nsupdate/setup.sh +++ b/bin/tests/system/nsupdate/setup.sh @@ -13,11 +13,6 @@ . ../conf.sh -# -# jnl and database files MUST be removed before we start -# -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/padding/setup.sh b/bin/tests/system/padding/setup.sh index 594ae98d9d..eb37490200 100644 --- a/bin/tests/system/padding/setup.sh +++ b/bin/tests/system/padding/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL ./clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/pending/setup.sh b/bin/tests/system/pending/setup.sh index bf57f90369..f16857c943 100644 --- a/bin/tests/system/pending/setup.sh +++ b/bin/tests/system/pending/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/pipelined/setup.sh b/bin/tests/system/pipelined/setup.sh index 49a642683d..b0cb266ec7 100644 --- a/bin/tests/system/pipelined/setup.sh +++ b/bin/tests/system/pipelined/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/rpz/setup.sh b/bin/tests/system/rpz/setup.sh index d1528e85ee..1fb9e0f84a 100644 --- a/bin/tests/system/rpz/setup.sh +++ b/bin/tests/system/rpz/setup.sh @@ -17,8 +17,6 @@ set -e . ../conf.sh -$SHELL clean.sh - for dir in ns*; do touch $dir/named.run nextpart $dir/named.run >/dev/null diff --git a/bin/tests/system/rpzrecurse/setup.sh b/bin/tests/system/rpzrecurse/setup.sh index 8d21434148..68615c4c19 100644 --- a/bin/tests/system/rpzrecurse/setup.sh +++ b/bin/tests/system/rpzrecurse/setup.sh @@ -17,8 +17,6 @@ set -e . ../conf.sh -$SHELL clean.sh - $PERL testgen.pl copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/rrl/setup.sh b/bin/tests/system/rrl/setup.sh index 49a642683d..b0cb266ec7 100644 --- a/bin/tests/system/rrl/setup.sh +++ b/bin/tests/system/rrl/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/rrsetorder/setup.sh b/bin/tests/system/rrsetorder/setup.sh index fbb1a38f9d..5d12d6ac95 100644 --- a/bin/tests/system/rrsetorder/setup.sh +++ b/bin/tests/system/rrsetorder/setup.sh @@ -13,7 +13,6 @@ . ../conf.sh -$SHELL clean.sh copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/runtime/setup.sh b/bin/tests/system/runtime/setup.sh index 1b0cfc2c6e..234f6489ba 100644 --- a/bin/tests/system/runtime/setup.sh +++ b/bin/tests/system/runtime/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns2/named1.conf.in ns2/named.conf copy_setports ns2/named-alt1.conf.in ns2/named-alt1.conf diff --git a/bin/tests/system/serve-stale/setup.sh b/bin/tests/system/serve-stale/setup.sh index 42e1ac8164..ad6073a793 100644 --- a/bin/tests/system/serve-stale/setup.sh +++ b/bin/tests/system/serve-stale/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named1.conf.in ns1/named.conf copy_setports ns3/named.conf.in ns3/named.conf copy_setports ns4/named.conf.in ns4/named.conf diff --git a/bin/tests/system/tcp/setup.sh b/bin/tests/system/tcp/setup.sh index 475f399048..1c587b5d39 100644 --- a/bin/tests/system/tcp/setup.sh +++ b/bin/tests/system/tcp/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf copy_setports ns3/named.conf.in ns3/named.conf diff --git a/bin/tests/system/transport-acl/setup.sh b/bin/tests/system/transport-acl/setup.sh index f726f7020d..c62497b30b 100644 --- a/bin/tests/system/transport-acl/setup.sh +++ b/bin/tests/system/transport-acl/setup.sh @@ -14,8 +14,6 @@ # shellcheck disable=SC1091 . ../conf.sh -$SHELL clean.sh - $SHELL "${TOP_SRCDIR}"/bin/tests/system/genzone.sh 2 >ns1/example.db copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/transport-change/setup.sh b/bin/tests/system/transport-change/setup.sh index f726f7020d..c62497b30b 100644 --- a/bin/tests/system/transport-change/setup.sh +++ b/bin/tests/system/transport-change/setup.sh @@ -14,8 +14,6 @@ # shellcheck disable=SC1091 . ../conf.sh -$SHELL clean.sh - $SHELL "${TOP_SRCDIR}"/bin/tests/system/genzone.sh 2 >ns1/example.db copy_setports ns1/named.conf.in ns1/named.conf diff --git a/bin/tests/system/tsig/setup.sh b/bin/tests/system/tsig/setup.sh index cd90cee785..4eed3fc582 100644 --- a/bin/tests/system/tsig/setup.sh +++ b/bin/tests/system/tsig/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf if $FEATURETEST --md5; then diff --git a/bin/tests/system/tsiggss/setup.sh b/bin/tests/system/tsiggss/setup.sh index 0353cb50ef..af9dcfe191 100644 --- a/bin/tests/system/tsiggss/setup.sh +++ b/bin/tests/system/tsiggss/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf key=$($KEYGEN -Cq -K ns1 -a $DEFAULT_ALGORITHM -b $DEFAULT_BITS -n HOST -T KEY key.example.nil.) diff --git a/bin/tests/system/zonechecks/setup.sh b/bin/tests/system/zonechecks/setup.sh index ffefaf3c92..7851686325 100644 --- a/bin/tests/system/zonechecks/setup.sh +++ b/bin/tests/system/zonechecks/setup.sh @@ -13,8 +13,6 @@ . ../conf.sh -$SHELL clean.sh - copy_setports ns1/named.conf.in ns1/named.conf copy_setports ns2/named.conf.in ns2/named.conf diff --git a/doc/dev/dev.md b/doc/dev/dev.md index b20159d918..1aa4e44eea 100644 --- a/doc/dev/dev.md +++ b/doc/dev/dev.md @@ -181,10 +181,6 @@ The following standard files are found in system test directories: - `tests.sh`: runs all the test cases. A non-zero return value results in R:FAIL -- `clean.sh`: run at the end to clean up temporary files, but only if the - test was completed successfully; otherwise the temporary files are left - in place for inspection. - - `ns[X]`: these subdirectories contain test name servers that can be queried or can interact with each other. (For example, `ns1` might be running as a root server, `ns2` as a TLD server, and `ns3` as a recursive