Commit graph

8154 commits

Author SHA1 Message Date
Michał Kępień
3752edb28e [rt46602] Assign default values to variables set by command line options and consistently treat them as integers
(cherry picked from commit fe51c34df3)
2018-02-22 11:14:39 +00:00
Michał Kępień
f47ed5ceb0 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4
(cherry picked from commit 1960de5530)
2018-02-22 11:14:39 +00:00
Michał Kępień
8f4770facf [rt46602] Re-add clean.sh invocation to catz/setup.sh
Commit 57aa7b60fd caused catz/setup.sh to no longer call clean.sh, which
results in the catz system test failing on subsequent runs if the first
run is interrupted or fails.

(cherry picked from commit 344d05063f)
2018-02-22 11:14:39 +00:00
Stephen Morris
8376949ea8 [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".

(cherry picked from commit 451b7309ca)
2018-02-22 11:14:39 +00:00
Stephen Morris
53ebdf225f [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.

(cherry picked from commit 462766cd76)
2018-02-22 11:14:32 +00:00
Stephen Morris
8513c4256b [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)

(cherry picked from commit fb78f82ba1)
2018-02-22 11:13:58 +00:00
Stephen Morris
4731a82eef [rt46602] Minor documentation changes
(cherry picked from commit cdf18a5009)
2018-02-22 11:13:58 +00:00
Stephen Morris
36136dec55 [rt46602] Correct way allow_query test produces RNDC output
(cherry picked from commit 7f2d41d4c4)
2018-02-22 11:13:58 +00:00
Stephen Morris
fb5974f902 [rt46602] Simplify the cleanup code
(cherry picked from commit 0855a8a6cc)
2018-02-22 11:13:58 +00:00
Stephen Morris
56ba2e822b [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.

(cherry picked from commit f606b17dfb)
2018-02-22 11:13:58 +00:00
Stephen Morris
3329876ffd [rt46602] Miscellenous small changes
(cherry picked from commit b88b508b22)
2018-02-22 11:13:58 +00:00
Stephen Morris
6602a31229 [rt46602] Handle echoing of multi-line output from tests
(cherry picked from commit 1f07c69f25)
2018-02-22 11:13:58 +00:00
Stephen Morris
7a5476173a [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.

(cherry picked from commit c12e06755f)
2018-02-22 11:13:58 +00:00
Stephen Morris
6917ea971a [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.

(cherry picked from commit 00bc29640b)
2018-02-22 11:13:57 +00:00
Stephen Morris
ff5349b3b3 [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.

(cherry picked from commit 244d1c30e2)
2018-02-22 11:13:57 +00:00
Stephen Morris
6bd944c9d0 [rt46602] Ensure that output sent to stderr is routed to the output file
(cherry picked from commit a5cf6bd12a)
2018-02-22 11:13:57 +00:00
Stephen Morris
6b5e5a2da2 [rt46602] Minor tidying up of documentation
(cherry picked from commit ce66752adc)
2018-02-22 11:13:57 +00:00
Stephen Morris
0afcdf658d [rt46602] Check prerequsities before network interfaces when running test.
(cherry picked from commit 549a6c6bc8)
2018-02-22 11:13:57 +00:00
Stephen Morris
5fc5b514e4 [rt46602] Minor formatting changes to Perl files
(cherry picked from commit f0f5b5f045)
2018-02-22 11:13:57 +00:00
Stephen Morris
88081b765e [rt46603] Provide echostart() and echoend() to output test start/end messages.
(cherry picked from commit c344e1d768)
2018-02-22 11:13:57 +00:00
Stephen Morris
50769a4a9e [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.

(cherry picked from commit 055e5be9fd)
2018-02-22 11:13:50 +00:00
Stephen Morris
0e2e251a0d [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.

(cherry picked from commit 3c25e5fa02)
2018-02-22 11:13:26 +00:00
Stephen Morris
61514873e9 [rt46602] Allocate 100 ports to each test
(cherry picked from commit edc7a18d84)
2018-02-22 11:13:04 +00:00
Stephen Morris
f10b6cfeb6 [rt46602] Pass port numbers to tests via environment variables
(cherry picked from commit f5d8f07900)
2018-02-22 11:12:57 +00:00
Stephen Morris
bf2992dbe6 [rt46602] Ensure parallel tests are in alphabetical order
(cherry picked from commit b2d90da0c2)
2018-02-22 11:12:33 +00:00
Stephen Morris
3952d4631e [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.

(cherry picked from commit af005cdbcf)
2018-02-22 11:12:23 +00:00
Stephen Morris
78494c3a4d [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.

(cherry picked from commit b24c2e11d8)
2018-02-22 11:11:51 +00:00
Stephen Morris
b01b553d72 [rt46602] Modify some test files to cope with the "-p" flag
(cherry picked from commit 517d990bd4)
2018-02-22 11:11:44 +00:00
Stephen Morris
a49afdaeb8 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.

(cherry picked from commit 32fe6f7682)
2018-02-22 11:11:12 +00:00
Stephen Morris
df1348ad2b [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.

(cherry picked from commit e0ff77f9d3)
2018-02-22 11:11:12 +00:00
Stephen Morris
58de0adbf6 [rt46602] Allow catz system test to run in parallel with other tests
(cherry picked from commit 07341693e1)
2018-02-22 11:11:12 +00:00
Stephen Morris
01eb4d7c97 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.

(cherry picked from commit 78f2b9ca01)
2018-02-22 11:11:00 +00:00
Stephen Morris
4f68e67e5c [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.

(cherry picked from commit e7429b124b)
2018-02-22 11:09:59 +00:00
Stephen Morris
ba694ca38c [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch
(cherry picked from commit e1a69e5d1f)
2018-02-22 11:08:42 +00:00
Ondřej Surý
a14bd48268 [rt46602] Allow parallel run of new dynamic port tests and the old ones
(cherry picked from commit 3b14d54fd7)
2018-02-22 11:08:05 +00:00
Ondřej Surý
cf7a86661a [rt46602] Put back original .gitignore files in the system tests
(cherry picked from commit caaec1ef6d)
2018-02-22 11:08:05 +00:00
Ondřej Surý
c5aa6ed87b [rt46602] Adjust rpzrecurse for dynamic port usage
Resolve conflict in run.sh during cherry-pick from an experimental branch.

(cherry picked from commit 2a435708e4)
2018-02-22 11:07:30 +00:00
Ondřej Surý
d37a52cc56 [rt46602] Add $SEDPORTS variable to help with @PORT@ and @CONTROLPORT@ replacement
(cherry picked from commit e6802badc0)
2018-02-22 11:02:14 +00:00
Ondřej Surý
a5c3077a9c [rt46602] Move most of ignored files to top of system tests directory
(cherry picked from commit d2c8286163)
2018-02-22 11:01:35 +00:00
Ondřej Surý
112355adb8 [rt46602] Fix output from rndc reload
(cherry picked from commit 53bade5ece)
2018-02-22 11:01:35 +00:00
Ondřej Surý
76dfb5c339 [rt46602] Add .gitignore to allow_query
(cherry picked from commit bc276f6c05)
2018-02-22 11:01:35 +00:00
Ondřej Surý
93034acb4d [rt46602] Utilize echo_i to allow mixed test output
(cherry picked from commit 7e3c96c867)
2018-02-22 11:01:29 +00:00
Ondřej Surý
b8e866d60f [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.

(cherry picked from commit 278c72fef6)
2018-02-22 11:01:08 +00:00
Ondřej Surý
bd5b36f7a4 [rt46602] Add echo_i function to print I:<testname>:string
(cherry picked from commit f7ac41757a)
2018-02-22 11:01:08 +00:00
Ondřej Surý
57adb949a4 [rt46602] Add serve-stale working files to .gitignore
(cherry picked from commit cba162e70e)
2018-02-22 11:01:07 +00:00
Ondřej Surý
1d10178e83 [rt46602] Convert allow_query to dynamic port
(cherry picked from commit 992a2bafec)
2018-02-22 11:01:01 +00:00
Ondřej Surý
0ee883d443 [rt46602] Change the system tests script to support running individual tests on different ports
Resolved conflict in run.sh.

(cherry picked from commit fb00f18005)
2018-02-22 10:58:33 +00:00
Mark Andrews
e0c9b5bd2b check that sed selects a non zero range of lines
(cherry picked from commit 146c16c06e)
2018-02-22 12:37:27 +11:00
Mark Andrews
ea29bec179 Merge branch '67-stabilize-cacheclean-system-test' into 'master'
Stabilize cacheclean system test

Closes #67

See merge request isc-projects/bind9!43

(cherry picked from commit 643c8c27ff)

01de79b3 Assign an index to each check in the cacheclean system test
aeea1faf Do not overwrite cache dumps
2bbff06d Wait until a cache dump completes instead of waiting for a fixed amount of time
ca1049b2 Improve the way cache contents are searched for "ns.flushtest.example"
2018-02-20 16:08:59 -05:00
Mark Andrews
f103cbee94 return failure count 2018-02-16 15:08:24 +11:00