- Use a common ns2/named.conf.j2 template for all the "#TN"
replacements. Instead of commenting out with sed, render the template
differently into ns/namedX.conf using variables.
- Keep the final ns2/named7.conf.j2 (formerly ns2/named2.conf.in) as a
separate template for readability due to significant differences.
(cherry picked from commit 6114444007)
- The ns3/named.conf.j2 uses has a "#BAD" section that is only included
after restart. Turn it into ns3/named2.conf.
- Since the original config is then restored, keep a copy of it as
ns3/named1.conf using a symlink, causing it to be rendered twice.
- Use jinja2 templates for test* files to render the port number instead
of calling copy_setports in load_db().
(cherry picked from commit c93733263a)
- Instead of strings to be replaced by sed, use proper jinja templates.
- ns3/named1.conf.j2 is basically a copy of the default config, because
it needs to be restored later in the test.
(cherry picked from commit efc302b0ad)
- Move ns1/named.conf.j2 to ns1/named2.conf.j2 and adjust the python
test to render this template.
- Convert remaining .in files to .j2 and handle the multiple configs.
(cherry picked from commit c46208ffdb)
- Rename named.conf.j2 to named3.conf.j2 and adjust the python test to
render this template.
- Handle the n2 and ns3 multiple configs as in other similar cases
(ns2/named1.conf.in was moved to ns2/named.conf.j2).
(cherry picked from commit 2592f0d961)
The test uses a second named2.conf template. It only differs by using
dnssec-validation yes. Render both named configs and swap them when
needed.
(cherry picked from commit 1da733a015)
- Merge ns*/statistics-channels.conf.in config snippets into
conditionally rendered section in ns*/named.conf.j2 files.
- Turn ns2/named.conf.in into ns2/named1.conf.j2 because it is used
later in the test to restore the original config.
- Symlink the ns2/named.conf.j2 ns2/named1.conf.j2 to pick a starting
config.
(cherry picked from commit cb88d1eda6)
- Rename ns1/named-fips.conf.in to ns1/named.conf.j2.
- Move MD5-specific stuff into ns1/named-md5.conf.j2 and include it
conditionally.
(cherry picked from commit 8db37ac31c)
- Change ns2 header into jinja2 template.
- Keep the various ns2 config files as non-templates, same for the
named.default.conf to be consistent.
- Symlink the ns2/named.default.conf as a jinja2 template to pick a
starting config. It is rendered as a template to avoid an error when
the test would overwrite a git-tracked file.
- Use jinja2 templates for the ns3 files, keep named1.conf around
because it's needed later in the test to restore the config. Symlink
it to `ns3/named.conf.j2` to select a default config.
(cherry picked from commit 16e9ac4710)
- Merge ns1/tls.options.in into ns1/named.conf.j2 and render it
conditionally. Also conditionally include the additional
ns1/tls.conf.j2 which is always rendered.
- Use multiple templates for ns7 and replace the copy_setports.
- Use jinja2 template for verylarge.in as well.
(cherry picked from commit 54c95d6966)
- Merge options-tls.conf into named.conf in ns2 and ns4 and render it
conditionally. Also conditionally include the additional
named-tls.conf which is always rendered.
(cherry picked from commit 9fc446b6f5)
- Merge options-tls.conf into named.conf in ns2 and ns4 and render it
conditionally. Also conditionally include the additional
named-tls.conf which is always rendered.
- Use multiple templates for ns3 and ns9 and replace the copy_setports.
- In ns3/named2.conf, use "root2.db" directly rather than replacing it
with "sed" later.
(cherry picked from commit 8e525f2154)
- Replace all named*.in files with jinja2 templates.
- Rename the config files to named.*.conf convention.
- Rename named.plain.in to named.plain.conf.j2 and symlink it as te
default config.
- Rename named.plainconf.in to named.plainlog.conf.j2 (there is a slight
difference from named.plain, despite the similar name)
(cherry picked from commit 3f953266fb)
- Replace named-altX.conf.in with namedX.conf.j2 to stick with the same
naming convention across the entire code base. Note than due to
named1.conf being the first (default) config, the numbers for the altX
are incremented.
- Turn alt9 into named7 to stick with the same number sequence. Adjust
the related file names accordingly.
(cherry picked from commit 7f3b0afb09)
Render the bad-tsig.db file using jinja2 template to get rid of
copy_setports.
Since the zone is using @ character, use the raw directive to avoid
interpreting it as a variable start.
(cherry picked from commit b23301ec55)
- Replace named*.conf.in files with jinja2 templates.
- When applying the files use plain cp command to copy the rendered
files.
(cherry picked from commit 0f37603b1a)
The following tests use multiple named configs. Previously, these have
been rendered with copy_setports in tests.sh when needed. Transform
these into jinja2 templates and render them during setup. In the tests,
the copy_setports invocations can be then replaced with a simple cp.
(cherry picked from commit 9d3279a542)
Use jinja2 templates instead of *.in templates for named.conf and remove
the copy_setports invocations from setup.sh which are no longer needed.
(cherry picked from commit fc10cb686d)
This allows rendering multiple named*.conf files using the jinja2
template engine at test start and then simply copying the required
config to named.conf as needed.
(cherry picked from commit d6d6db52e3)
Pytest sets the test names as `test_foo` and the old test runner spits
out `bin/tests/system/foo`.
Normalize this to match the new test runner.
Merge branch 'stepan/match-pytest-junit-names' into 'bind-9.20'
See merge request isc-projects/bind9!11085
This is a fixup for MR !11345. The variable for --extended-ds-digest was
accidentally backported - 9.20 doesn't have this feature check.
Merge branch 'nicki/featuretest-env-backport-fixup' into 'bind-9.20'
See merge request isc-projects/bind9!11357
A catalog zone's member zone could fail to load in some rare cases, when
the internally generated zone configuration string was exceeding 512
bytes. That condition only was not enough for the issue to arise, but it
was a necessary condition. This could happen, for example, if the catalog
zone's default primary servers list contained a large number of items.
This has been fixed.
Closes#5658
Backport of MR !11281
Merge branch 'backport-5658-dns_catz_generate_zonecfg-bug-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!11349
Use a member zone with a long list of primaries with long key
names to trigger the issue that was fixed by the previous commit.
(cherry picked from commit 2622140482)
The dns_catz_generate_zonecfg() function generates a zone configuration
string to use with a new catalog zone member zone. The buffer for the
string is 512 bytes initially (ISC_BUFFER_INCR), but can be reallocated
when required, when using corresponding isc_buffer functions like
isc_buffer_reserve(), isc_buffer_putstr(), isc_buffer_copyregion(), etc.
However, the dns_name_totext() function, which expects the buffer as an
argument, doesn't automatically resize it if the name doesn't fit there,
but instead just returns ISC_R_NOSPACE.
The chance of this occurring increases when the configuration string is
large due to, for example, long zone name, long list of primary servers
which have keys configured and/or TLS configured.
Use dns_name_format() accompanied with isc_buffer_putstr() instead of
dns_name_totext().
(cherry picked from commit 684d7e008a)
The dns module does have it.
Backport of MR !11348
Merge branch 'backport-mnowak/enable-selftest-system-test-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!11350
The purpose of these variables is to be able to detect feature support
without calling feature-test. This becomes useful when detecting feature
support in jinja2 templates.
Backport of MR !11316
Merge branch 'backport-nicki/featurest-system-test-env-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!11345
The purpose of these variables is to be able to detect feature support
without calling feature-test. This becomes useful when detecting feature
support in jinja2 templates.
(cherry picked from commit 19af19b31c)
Improve and unify the handling of regular expressions when searching in logs, files and command output in system tests.
- Use `Re()` for constructing regular expressions, which is an imported shorthand for `re.compile()` (imported as `from re import compile as Re`
- Add new `isctest.text.Text` interface which is a text wrapper that supports the `in` operator for line matching operation for both strings and regular expressions, e.g.:
- `assert "running" in ns1.log`
- `assert Re("a.example..*10.0.0.1") in response.out`
- Use the new `isctest.text.Text` for:
- `isctest.run.cmd()` output, where `.out` and `.err` can be used for stdout and stderr contents
- `NamedInstance.log` rather than the previous log interface (`.expect()` and `.prohibit()` is no longer available or needed. The `in` operator along with an `assert` statement can be used now instead.)
- `NamedInstance.rndc()` output, which returns identical output as `isctest.run.cmd()`
Backport of MR !11054
Merge branch 'backport-nicki/pytest-grep-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!11342
To unify the command handling, utilize EnvCmd() to handle rndc commands:
1. Remove isctest.rndc abstractions. They were intended for an upcoming
python-only implementation. A couple of years later, it doesn't seem
to be coming any time soon, so let's stick with the interface that
makes sense today, i.e. use the same command handling interface
everywhere.
2. Remove the specialized rndc.log in favor of the generic logging
already implemented by isctest.run.cmd(). I believe the cause of the
many rndc(log=False) invocations was that nobody wanted this extra
file. Yet, logging everything by default makes sense for debugging,
unless there's a good reason not to. In almost all cases, logging was
switched to the default (enabled).
3. With the NamedInstance.rndc() call now returning CmdResult rather
than combined stdout+stderr string, adjust all the invocations to use
`.out` or `.err` as necessary.
4. Replace some manual rndc invocation and its base argument
construction with the standardized nsX.rndc() call.
5. In cases where rndc is expected to fail, utilize
raise_on_exception=False and check the `.rc` from the result, rather
than handling an exception.
6. In addzone/tests_rndc_deadlock.py, refactor the test slightly to
avoid using EnvCmd() entirely to avoid spamming the logs. This test
calls rndc in a loop from multiple threads and such test case is an
exception which doesn't warrant changing the `isctest.run.cmd()`
implementation.
(cherry picked from commit f33e2b6d87)
A generic helper that calls the environment-specified binaries in a
developer-friendly manner, i.e. passing arguments as strings rather than
having to split them first.
The isctest.run.cmd() remains as the basis which provides a clean and
robust interface, while the isctest.run.EnvCmd() can be used as a
convenient wrapper for tests, or when there are some shared default
parameters.
The isctest.run.Dig() is superseded with the isctest.run.EnvCmd(). In
the future, we might revisit adding Dig() or command-specific helpers
again, but it probably only makes sense if they offer command-aware
attributes / methods, rather than just being shortcuts to
isctest.run.EnvCmd().
(cherry picked from commit ff613a72d7)
Refactor the file handling to write to a file directly when calling
isctest.run.cmd().
Refactor the existing code to use CmdResult rather than out and err
separately.
(cherry picked from commit 9bad9491a1)
When commands are executed using the isctest.run.cmd() command, allow
the output to be Grep-able like logs and text files.
(cherry picked from commit 4b6a86b029)
Add a new Grep-like interface which can be used for searching for
regular expressions in files. Replace the prior LogFile used for named
logs with the new TextFile interface.
(cherry picked from commit 7743bab5fc)
Add a new module for working with text and keep the isctest.log.watchlog
module focused on its purpose. Move LogFile and LineReader into the new
module. Add compile_pattern() helper which will be useful in subsequent
commits.
(cherry picked from commit be6bae2a75)
It's a fairly common pattern to use regular expression in our tests.
Instead of using the fairly verbose re.compile(), import that function
as Re() instead to allow for more brevity in the test syntax.
(cherry picked from commit ac7127d620)
Avoid repeating the .decode("utf-8") snippet when processing command
output and provide a helper instead, which leads to more concise code.
(cherry picked from commit ac998da3f6)
When creating an NSEC3 opt-out chain, a node in the chain could be removed too soon, causing the previous NSEC3 being unable to be found, resulting in invalid NSEC3 records to be left in the zone. This has been fixed.
Closes#5671
Backport of MR !11328
Merge branch 'backport-5671-fix-dbiterator-prev-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!11340
