When a test is re-run by the flaky plugin, the TestReport outcomes
collected in the pytest_runtest_makereport() hook should be overriden.
Each of the setup/call/teardown phases is reported again and since we
care about the overall outcome, their respective results should be
overriden so that only the outcome from the final test (re)run gets
reported.
Prior to this change, it lead to a situation where an extra_artifact
generated during the test might be ignored. This was caused because the
check was skipped, since the test was incorrectly considered as "failed"
in the case where the test would fail on the first run, but pass on a
subsequent flaky rerun.
(cherry picked from commit b66fb31dcb)
The artifact lists in clean.sh and extra_artifacts might be slightly
different. The list was updated for each test to reflect the current
state.
(cherry picked from commit 7c259fe254)
Prior to introducing the pytest runner, clean.sh files were used as a
list of files that the test is expected to leave around as artifacts and
check that no extra files were created.
With the pytest runner, those scripts are no longer used, but the
ability to detect extraneous files is still useful. Add a new
"extra_artifacts" mark which can be used for the same purpose.
(cherry picked from commit 3a9f4edddc)
Configuration files in system tests which require some variables (e.g.
port numbers) filled in during test setup, can now use jinja2 templates
when `jinja2` python package is available.
Any `*.j2` file found within the system test directory will be
automatically rendered with the environment variables into a file
without the `.j2` extension by the pytest runner. E.g.
`ns1/named.conf.j2` will become `ns1/named.conf` during test setup. To
avoid automatic rendering, use `.j2.manual` extension and render the
files manually at test time.
New `templates` pytest fixture has been added. Its `render()` function
can be used to render a template with custom test variables. This can be
useful to fill in different config options during the test. With
advanced jinja2 template syntax, it can also be used to include/omit
entire sections of the config file rather than using `named1.conf.in`,
`named2.conf.in` etc.
(cherry picked from commit 60e118c4fb)
The pytest collection mechanism has been overhauled in pytest 8.0.0,
resulting in a different node tree when collecting the tests. Ensure the
paths / names we're using that are derived from the node tree are
consistent across different pytest versions.
Particularly, this has affected the convenience symlink name (which is
supposed to be in the form of e.g. dns64_sh_dns64 for the dns64 module
and tests_sh_dns64.py module) and the test name that's logged at the
start of the test, which is supposed to include the system test
directory relative to the root system test directory as well as the
module name (e.g. dns64/tests_sh_dns64.py).
Related https://github.com/pytest-dev/pytest/issues/7777
(cherry picked from commit 7118cbed98)
For dynamic zones that do not set inline-signing explicitly, add a
warning that the default value for inline-signing has changed. Dynamic
zones that want to be able to reuse the zone (and not trigger a full
resign) should explicitly configure "inline-signing no;".
Enforcing pylint standards and default for our test code seems
counter-productive. Since most of the newly added code are tests or is
test-related, encountering these checks rarely make us refactor the code
in other ways and we just disable these checks individually. Code that
is too complex or convoluted will be pointed out in reviews anyways.
(cherry picked from commit 7639c58c48)
CookieOption with new .server/.client attributes (rather than .data) was
added to dnspython. Adjust the code to use the new attributes if
available and fall back to the old code for dnspython<2.7.0
compatibility.
(cherry picked from commit 0d90b13646)
In a multi-signer setup, removing DNSKEY records from the zone should
not be treated as a key that previously exists in the keyring, thus
blocking the keymgr. Add a test case to make sure.
(cherry picked from commit 5f552293d7)
Test that if a key to be purged is in the keyring, it does not
prevent the keymgr from running. Normally a key that is in the keyring
should be available again on the next run, but that is not true for
a key that can be purged.
In addition, fix some wait_for_log calls, by adding the missing
'|| ret=1' parts.
(cherry picked from commit a2317425bc)
Some test cases were working but for the wrong reasons. These started
to fail when I implemented the first approach for #4763, where the
existence of a DNSKEY together with an empty keyring is suspicious and
would prevent the keymgr from running.
These are:
1. kasp: The multisigner-model2.kasp zone has ZSKs from other providers
in the zone, but not yet its own keys. Pregenerate signing keys and
add them to the unsigned zone as well.
2. kasp: The dynamic-signed-inline-signing.kasp zone has a key generated
and added in the raw version of the zone. But the key file is stored
outside the key-directory for the given zone. Add '-K keys' to the
dnssec-keygen command.
(cherry picked from commit d1e263ef13)
Prior to running the keymgr, first make sure that existing keys
are present in the new keylist. If not, treat this as an operational
error where the keys are made offline (temporarily), possibly unwanted.
(cherry picked from commit 5fdad05a8a)
In this specific case the key files are temporary unavailable, for
example because of an operator error, or a mount failure). In such
cases, BIND should not try to roll over these keys.
(cherry picked from commit a3afbd9d6f)
This commit ensures that the port is set before attempting a UDP
query. Before that a situation could appear when previous query have
completed over a different transport (that uses a dedicated port) and
then a UDP query will be attempted over the port of the previous
transport.
(cherry picked from commit e390ed4421)
The ans4 server wasn't shutting down cleanly sometimes. Check that
read returns the expected value for the message length or exit read
loop.
(cherry picked from commit 21baad0a8f)
The shutdown system test sends queries when named is shutting down, not
in an attempt to get answers but to destabilize the server into a crash.
With isctest.query.udp() defaulting to try up to ten times with a
ten-second timeout to get a response we don't care about from a likely
terminated server, we make the test run much longer than needed because
of retries and long timeouts.
(cherry picked from commit 463ab2f3f5)
The 'bin/tests/system/resolver.c' tool used in the resolver system
test uses the 'dns_client_setservers()' function, which sets up a
resolution in 'forward only' mode. Since a bug was just fixed in
'fctx_getaddresses()', two expected failures in the resolver system
test now fail with a different failure message, because
'fctx_getaddresses()' returns 'ISC_R_FAILURE' instead of
'res->quotaresp[dns_quotatype_server]', which is 'DNS_R_SERVFAIL'
by default. Change the expected failure message.
Check that the fix in the previous commit works and that the
'ServerQuota' counter in the statistics channel is still unset
after a SERVFAIL result in a 'forward only' zone.
(cherry picked from commit 81b3c5d908)
Silence Coverity CID 468757 and 468767 (DATA RACE read not locked)
by converting dnssec-signzone to use atomics for statistics counters
rather than using a lock. This should be marginally faster than
using the lock as well when statistics are requested.
(cherry picked from commit 473cbd4e87)
This change allows fallback from an IXFR failure to AXFR when the
reason is DNS_R_TOOMANYRECORDS. This is because this error condition
could be temporary only in an intermediate version of IXFR
transactions and it's possible that the latest version of the zone
doesn't have that condition. In such a case, the secondary would never
be able to update the zone (even if it could) without this fallback.
This fallback behavior is particularly useful with the recently
introduced max-records-per-type and max-types-per-name options:
the primary may not have these limitations and may temporarily
introduce "too many" records, breaking IXFR. If the primary side
subsequently deletes these records, this fallback will help recover
the zone transfer failure automatically; without it, the secondary
side would first need to increase the limit, which requires more
operational overhead and has its own adverse effect.
This change also fixes a minor glitch that DNS_R_TOOMANYRECORDS wasn't
logged in xfrin_fail.
(cherry picked from commit 7289090683)
When the TCP test is run on the busy server, the server might take a
while to wind the server down because it might still be processing all
that 300k invalid XFR requests.
Increate the rncd wait time to 120 seconds, the SIGTERM time to 300
seconds, and reduce the time to wait for ans servers from 1200 second
to just 120 seconds.
(cherry picked from commit d971472321)
