diff --git a/bin/tests/system/README b/bin/tests/system/README index 8ecbe99fcb..c27074cbbe 100644 --- a/bin/tests/system/README +++ b/bin/tests/system/README @@ -67,7 +67,7 @@ Running the Tests === The tests can be run individually using the following command: - sh run.sh [flags] + sh run.sh [flags] [] e.g. @@ -75,8 +75,12 @@ e.g. Optional flags are: - -p Sets the range of ports used by the test. If not - specified, the test will use ports 5300 to 5309. + -p Sets the range of ports used by the test. A block of 10 + ports is available for each test, the number given to the + "-p" switch being the number of the start of that block + (e.g. "-p 7900" will mean that the test is able to use + ports 7900 through 7909). If not specified, the test will + use ports 5300 to 5309. -n Noclean - do not remove the output files if the test completes successfully. By default, files created by the test are deleted if it passes; they are not deleted if the @@ -87,17 +91,27 @@ Optional flags are: Python or Perl) that feature test-specific behavior. The servers are automatically started before the test is run and stopped after it ends. This flag leaves them running - at the end of the test. + at the end of the test, so that additional queries can be + sent by hand. To stop the servers afterwards, use the + command "sh stop.sh ". -d Arguments to the "date" command used to produce the start and end time of the tests. For example, the switch -d "+%Y-%m-%d:%h:%M:%s" - would cause the "S" and "E" messages (see below) to have the - date looking like "2017-11-23:16:06:32" instead of the + would cause the "S" and "E" messages (see below) to have + the date looking like "2017-11-23:16:06:32" instead of the default "Thu, 23 Nov 2017 16:06:32 +0000". +Arguments are: + + test-name Mandatory. The name of the test, which is the name of the + subdirectory in bin/tests/system holding the test files. + + test-arguments Optional arguments that are passed to each of the test's + scripts. + To run all the system tests, type either: sh runall.sh [numproc] @@ -110,7 +124,7 @@ When running all the tests, the output is sent to the file systests.output (in the bin/tests/system) directory. The "numproc" option specifies the maximum number of tests that can run -simultaneously. The default is 1, which means that all the test run +simultaneously. The default is 1, which means that all of the tests run sequentially. If greater than 1, up to "numproc" tests will run simultaneously. (Each will use a unique set of ports, so there is no danger of them interfering with one another.) @@ -198,7 +212,7 @@ files produced by the tests should be deleted first. Deletion of files produced by an individual test can be done with the command: - (cd testname ; sh clean.sh) + sh clean.sh Deletion of the files produced by the set of tests (e.g. after the execution of "runall.sh") can be deleted by the command: @@ -253,29 +267,28 @@ ans Like ns[X], but these are simple mock name servers implemented in Port Usage --- In order for the tests to run in parallel, each test requires a unique set of -ports. These are specified by the "-p" option passed to "run.sh". This option -is then passed to each of the test control scripts listed above. +ports. These are specified by the "-p" option passed to "run.sh", which sets +environment variables that the scripts listed above can reference. The convention used in the system tests is that the number passed is the start of a range of 10 ports. The test is free to use the ports as required, although present usage is that the lowest port is used as the query port and -the highest is used as the control port. This is reinforced by the script -getopts.sh: if used to parse the "-p" option (see below), the script sets the -following shell variables: +the highest is used as the control port. This is reinforced by the environment +variables set by run.sh. These are: - port Number to be used for the query port. - controlport Number to be used as the RNDC control port. - aport1 - aport8 Eight port numbers that the test can use as needed. + PORT Number to be used for the query port. + CONTROLPORT Number to be used as the RNDC control port. + EXTRAPORT1 - EXTRAPORT8 Eight port numbers that can be use as needed. + +Two other environment variables are defined: + + LOWPORT The lowest port number in the range. + HIGHPORT The highest port number in the range. When running tests in paralel (i.e. giving a value of "numproc" greater than 1 in the "make" or "runall.sh" commands listed above), it is guaranteed that each test will get a set of unique port numbers. -In addition, the "getopts.sh" script also defines the following symbols: - - portlow Lowest port number in the range. - porthigh Highest port number in the range. - Writing a Test --- @@ -285,51 +298,19 @@ nameserver instances) to run. Certain expectations are put on each script: General --- -Each of the four scripts will be invoked with the command +1. Each of the four scripts will be invoked with the command - sh