System test used to have sequential system tests, which can't run in
parallel with the rest of system tests. As there are no such tests
anymore the underlying infrastructure can be dropped.
"parallel.sh" script was used on Windows to run system tests in
parallel. Since Windows support was removed from BIND 9, the script is
not needed anymore.
testsummary.sh was not updated after build system rewrite to Autotools,
and needs to be fixed to produce test summary and core dump, assertion
failures, and ThreadSanitizer reports.
Given that all of this is provided by Autotools and run.sh already,
there's little use to testsummary.sh script and should be dropped.
runall.sh was mainly used on Windows and as it's support was removed
from the "main" branch the script is not needed anymore.
Also, remove bin/tests/system/README text on running multiple system
test suites simultaneously with runall.sh as that support was not
present in the script anyway.
bin/tests/system/setup.sh just executes setup.sh script of a particular
system test in the directory of the system test. This does not seems to
be useful enough to maintain it.
stopall.sh script takes almost 2 minutes to go thru all test
subdirectories (due to a sleep in stop.pl) and does not seems to be
efficient way to stop manually started tests.
The keyfromlabel system ECDSA tests sometimes fail. When this happens
the ZSK and KSK key id values differ by 1, which is an indication that
the same key is used for both DNSKEY records.
When the private key is retrieved with 'ENGINE_load_private_key()', the
public key is already set. But sometimes that key differs from the key
which was retrieved with 'ENGINE_load_public_key()'.
The libp11 source code uses id to find the key and without IDs all the
keys are "equal", so it is returning the first key in the array of the
enumerated keys instead of the matching key. In our test we didn't use
'--id', just '--label'. With this change, the system test should no
longer fail intermittently.
Note this is only an issue for ECDSA keys, not RSA keys.
Add missing system test for dnssec-keyfromlabel. Test for various
algorithms that we can generate key files from a key that is stored in a
HSM, and that those keys can be used for signing with dnssec-signzone.
- Removed all code that only runs under CYGWIN, and made all
code that doesn't run under CYGWIN non-optional.
- Removed the $TP variable which was used to add optional
trailing dots to filenames; they're no longer optional.
- Removed references to pssuspend and dos2unix.
- No need to use environment variables for diff and kill.
- Removed uses of "tr -d '\r'"; this was a workaround for
a cygwin regex bug that is no longer needed.
Commit c787a539d2 fixed a certain class of
intermittent system test failures caused by named instances unable to
restart. The root cause was bin/tests/system/stop.pl returning without
waiting for a named instance to remove its lock file.
Later on, it turned out that the above change causes other issues on
Windows due to the way named handles signals on that platform. Commit
761ba4514f intended to address those
issues by making the server_lock_file() subroutine in
bin/tests/system/stop.pl return an empty value on Windows, in order to
prevent the script for waiting for lock file cleanup on that platform.
Note, however, that Windows detection in that subroutine is limited to
checking whether the CYGWIN environment variable is set.
While that environment variable was not set on Unix-like systems before
commit 761ba4514f, another commit
(a33237f070, merged a few weeks later)
changed that by setting the CYGWIN environment variable to an empty
value on Unix-like systems. This made the defined($ENV{'CYGWIN'}) check
in server_lock_file() return true, inadvertently preventing
bin/tests/system/stop.pl from waiting for lock file removal before
exiting on Unix-like systems and therefore reintroducing the original
issue.
Fix by making server_lock_file() only return an empty value when the
CYGWIN environment variable is set to a non-empty value (which is what
bin/tests/system/conf.sh.win32 does). Adjust a similar check in the
pid_file_exists() subroutine in the same way for consistency.
The echo_*() and cat_*() functions in bin/tests/system/conf.sh.common
call the "read" builtin command without specifying the field separator
to use. This results in leading whitespace getting stripped from each
line of the texts passed to those functions, which mangles e.g. pytest
output, hindering test failure troubleshooting.
Address by setting IFS to an empty value for the "read" calls used in
the aforementioned helper functions.
The bin/tests/system/start.pl script truncates the named.run file for a
given named instance unless it is invoked with the --restart
command-line option. Ever since Python-based tests were introduced,
bin/tests/system/run.sh may start named instances used by a given system
test multiple times within a single run, causing the
bin/tests/system/start.pl script to truncate some of the log files
written during the test. This makes troubleshooting certain test
failures hard or even impossible.
Fix by calling bin/tests/system/start.pl with the --restart command-line
option for every start_servers() invocation except the first one.
When failure is expected, the `rndc` command in the catz system test
is being called directly instead of using a function, i.e.:
$RNDC -c ../common/rndc.conf -s 10.53.0.2 -p 9953 reconfig \
> /dev/null 2>&1 && ret=1
... instead of:
rndccmd 10.53.0.2 reconfig && ret=1
This is done to suppress messages like "lt-rndc: 'reconfig' failed:
failure" appearing in the message log of the test, because failure
is actually expected, and the appearance of that message can be
confusing.
The port value used in this case is not correct, making the
`rndc reload` command to fail. This error was not detected earlier
only because the failure of the command is actually expected, but
the failure happens for a "wrong" reason, and the test still passes.
Fix the error by using the existing variable instead of the fixed
number.
Test the view reverting code by introducing a faulty dlz configuration
in named.conf and using `rndc reconfig` to check if named handles the
situation correctly.
We use "dlz" because the dlz processing code is located in an ideal
place in the view configuration function for the test to cover the
view reverting code.
This test is specifically added to the catz system test to additionally
cover the catz reconfiguration during the mentioned failed
reconfiguration attempt.
Sometimes the serving a query or two might fail in the test due to the
listeners not being reinitialised on time. This commit makes the test
suite to wait for reconfiguration message in the log file to detect
the time when the reconfiguration request completed.
gnutls-cli is tricky to script around as it immediately closes the
server connection when its standard input is closed. This prevents
simple shell-based I/O redirection from being used for capturing the DNS
response sent over a TLS connection and the workarounds for this issue
employ non-standard utilities like "timeout".
Instead of resorting to clever shell hacks, reimplement the relevant
check in Python. Exit immediately upon receiving a valid DNS response
or when gnutls-cli exits in order to decrease the test's run time.
Employ dnspython to avoid the need for storing DNS queries in binary
files and to improve test readability. Capture more diagnostic output
to facilitate troubleshooting. Use a pytest fixture instead of an
Autoconf macro to keep test requirements localized.
1. 10 seconds is an unfortunate pick because that reintroduces the
problem described in commit 5307bf64 (for an earlier check).
Change the +tries=3 +timeout=10 to +tries=2 +time=15, so that we
minimize the risk of dig missing any responses sent by the server in
the first 15 seconds while also increasing our chances of the
response arriving in time on machines under heavy load and allowing
it a single retry in case things go awry.
2. The comment about TCP above was misleading: as painfully proven by
GitLab CI, using TCP is no guarantee of receiving a response in a
timely manner. It may help a bit, but it is certainly not a 100%
reliable solution.
Change the dig invocation to just use UDP like in the two prior
tests for consistency (and revise that comment accordingly).
The resolver system tests was exhibiting often intermitten failures,
increase the timeout from default 5 second to 10 seconds to give the dig
more leeway for providing an answer.
This commit converts the license handling to adhere to the REUSE
specification. It specifically:
1. Adds used licnses to LICENSES/ directory
2. Add "isc" template for adding the copyright boilerplate
3. Changes all source files to include copyright and SPDX license
header, this includes all the C sources, documentation, zone files,
configuration files. There are notes in the doc/dev/copyrights file
on how to add correct headers to the new files.
4. Handle the rest that can't be modified via .reuse/dep5 file. The
binary (or otherwise unmodifiable) files could have license places
next to them in <foo>.license file, but this would lead to cluttered
repository and most of the files handled in the .reuse/dep5 file are
system test files.
Change RSASHA1 to $DEFAULT_ALGORITHM to be FIPS compliant.
There is one RSASHA1 occurence left, to test that dynamically adding an
NSEC3PARAM record to an NSEC-only zone fails.
Update the autosign system test with new expected behavior.
The 'nozsk.example' zone should have its expired zone signatures
deleted and replaced with signatures generated with the KSK.
The 'inaczsk.example' zone should have its expired zone signatures
deleted and replaced with signatures generated with the KSK.
In both scenarios, signatures are deleted, not retained, so the
"retaining signatures" warning should not be logged.
Furthermore, thsi commit fixex a test bug where the 'awk' command
always returned 0.
Finally, this commit adds a test case for an offline KSK, for the zone
'noksk.example'. In this case the expired signatures should be retained
(despite the zone being bogus, but resigning the DNSKEY RRset with the
ZSK won't help here).
All signed zone files present in bin/tests/system/inline/ns8 should
contain the unsigned serial number in the raw-format header. Add a
check to ensure that is the case. Extend the dnssec-signzone command
line in ns8/sign.sh with the -L option to allow the zones initially
signed there to pass the newly added check. Add another zone to the
configuration for the ns8 named instance to ensure the check also passes
when multiple zones are inline-signed by a single named instance.
This commit removes unused listen-on statements from the ns3 instance
in order to reduce the startup time. That should help with occasional
system test initialisation hiccups in the CI which happen because the
required instances cannot initialise in time.
Due to the fact that the primary nameserver creates a lot of TLS
contexts, its reconfiguration could take too much time on the CI,
leading to spurious test failures, while in reality it works just
fine.
This commit adds a separate instance for this test which does not use
ephemeral keys (these are costly to generate) and creates minimal
amount of TLS contexts.
This sets as many server options as possible at once to detect
cut-and-paste bugs when implementing new server options in peer.c.
Most of the accessor functions are similar and it is easy to miss
updating a macro name or structure element name when adding new
accessor functions.
checkconf/setup.sh is there to minimise the difference to branches
with optional server options where the list is updated at runtime.
