Commit graph

39352 commits

Author SHA1 Message Date
Nicki Křížek
347db7c40d Replace clean.sh files with extra_artifacts mark
The artifact lists in clean.sh and extra_artifacts might be slightly
different. The list was updated for each test to reflect the current
state.

(cherry picked from commit 7c259fe254)
2024-11-11 15:04:52 +01:00
Michał Kępień
ddb75fc4d8 Add pytest fixture for checking test artifacts
Prior to introducing the pytest runner, clean.sh files were used as a
list of files that the test is expected to leave around as artifacts and
check that no extra files were created.

With the pytest runner, those scripts are no longer used, but the
ability to detect extraneous files is still useful. Add a new
"extra_artifacts" mark which can be used for the same purpose.

(cherry picked from commit 3a9f4edddc)
2024-11-11 15:04:52 +01:00
Mark Andrews
b4a07e6426 [9.18] rem: nil: Remove named_g_sessionkey and named_g_sessionkeyname
Remove named_g_sessionkey and named_g_sessionkeyname as they are declared and cleaned up but otherwise are unused.

Closes #5023

Backport of MR !9720

Merge branch 'backport-5023-remove-named_g_sessionkey-as-it-is-unused-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9726
2024-11-07 05:15:54 +00:00
Mark Andrews
1affa9c2ac Remove named_g_sessionkey and named_g_sessionkeyname
They are only declared and cleaned up but otherwise unused.

(cherry picked from commit f70ff727ec)
2024-11-07 02:18:52 +00:00
Michal Nowak
0cd9476a13 [9.18] chg: test: Rewrite rsabigexponent system test to pytest
Backport of MR !9157

Merge branch 'backport-mnowak/pytest_rewrite_rsabigexponent-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9706
2024-10-31 19:15:10 +00:00
Michal Nowak
45dd10f5a1 Rewrite rsabigexponent system test to pytest
(cherry picked from commit cacff68e78)
2024-10-31 18:15:20 +00:00
Michal Nowak
020cad26aa [9.18] fix: doc: Remove the CHANGES file
With the introduction of the generated changelog, the CHANGES file
became a symlink to doc/arm/changelog.rst. After the changes made in
!9549, the changelog file transitioned from being a wholly generated
file to one that includes versioned changelog files, which are
themselves generated. However, while implementing !9549, we overlooked
that the CHANGES file is copied to a release directory on an FTP server
and contains just "include" directives, not the changelog itself.
Therefore, in the same fashion as the "RELEASE-NOTES*.html" file, create
a "CHANGELOG*.html" file that redirects to the Changelog appendix of the
ARM.

Closes #5000

Backport of MR !9690

Merge branch 'backport-5000-provide-correct-changelog-on-ftp-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9704
2024-10-31 18:12:17 +00:00
Michal Nowak
02054fcd73
Fix changelog history header
(cherry picked from commit 9750f22e3a)
2024-10-31 17:53:13 +01:00
Michal Nowak
37fbcedfaf Remove the CHANGES file
With the introduction of the generated changelog, the CHANGES file
became a symlink to doc/arm/changelog.rst. After the changes made in
!9549, the changelog file transitioned from being a wholly generated
file to one that includes versioned changelog files, which are
themselves generated. However, while implementing !9549, we overlooked
that the CHANGES file is copied to a release directory on an FTP server
and contains just "include" directives, not the changelog itself.
Therefore, in the same fashion as the "RELEASE-NOTES*.html" file, create
a "CHANGELOG*.html" file that redirects to the Changelog appendix of the
ARM.

(cherry picked from commit e40bd273e4)
2024-10-31 16:47:03 +00:00
Nicki Křížek
fa2ff6b690 [9.18] new: dev: Support jinja2 templates in pytest runner
Configuration files in system tests which require some variables (e.g.
port numbers) filled in during test setup, can now use jinja2 templates
when `jinja2` python package is available.

Any `*.j2` file found within the system test directory will be
automatically rendered with the environment variables into a file
without the `.j2` extension by the pytest runner. E.g.
`ns1/named.conf.j2` will become `ns1/named.conf` during test setup. To
avoid automatic rendering, use `.j2.manual` extension and render the
files manually at test time.

New `templates` pytest fixture has been added. Its `render()` function
can be used to render a template with custom test variables. This can be
useful to fill in different config options during the test. With
advanced jinja2 template syntax, it can also be used to include/omit
entire sections of the config file rather than using `named1.conf.in`,
`named2.conf.in` etc.

Closes #4938

Backport of MR !9587

Merge branch 'backport-4938-use-jinja2-templates-in-system-tests-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9700
2024-10-31 13:34:12 +00:00
Nicki Křížek
89866d148a Support jinja2 templates in pytest runner
Configuration files in system tests which require some variables (e.g.
port numbers) filled in during test setup, can now use jinja2 templates
when `jinja2` python package is available.

Any `*.j2` file found within the system test directory will be
automatically rendered with the environment variables into a file
without the `.j2` extension by the pytest runner. E.g.
`ns1/named.conf.j2` will become `ns1/named.conf` during test setup. To
avoid automatic rendering, use `.j2.manual` extension and render the
files manually at test time.

New `templates` pytest fixture has been added. Its `render()` function
can be used to render a template with custom test variables. This can be
useful to fill in different config options during the test. With
advanced jinja2 template syntax, it can also be used to include/omit
entire sections of the config file rather than using `named1.conf.in`,
`named2.conf.in` etc.

(cherry picked from commit 60e118c4fb)
2024-10-31 14:01:12 +01:00
Nicki Křížek
6d00009c17 [9.18] chg: ci: Make changelog audience mandatory
Backport of MR !9628

Merge branch 'backport-nicki/harazd-enforce-mr-title-audience-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9697
2024-10-29 16:12:39 +00:00
Nicki Křížek
c8eec28cde Make changelog audience mandatory
Use a stricter hazard check which ensures the audience tag is present in
the MR title and is one of the known values. This prevents siuations
where incorrect audience is accidentally used, resulting in a missing
changelog entry or a release note.

(cherry picked from commit cdb93bcbd4)
2024-10-29 16:03:37 +00:00
Nicki Křížek
f63b2304a1 [9.18] fix: test: Make system tests compatible with pytest 8.0.0+
Make system tests symlinks and logged test names consistent across pytest versions.

Backport of MR !9071

Merge branch 'backport-nicki/pytest-v8-compat-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9686
2024-10-29 15:20:36 +00:00
Nicki Křížek
2a0c3c8c14 Make system tests compatible with pytest 8.0.0+
The pytest collection mechanism has been overhauled in pytest 8.0.0,
resulting in a different node tree when collecting the tests. Ensure the
paths / names we're using that are derived from the node tree are
consistent across different pytest versions.

Particularly, this has affected the convenience symlink name (which is
supposed to be in the form of e.g. dns64_sh_dns64 for the dns64 module
and tests_sh_dns64.py module) and the test name that's logged at the
start of the test, which is supposed to include the system test
directory relative to the root system test directory as well as the
module name (e.g. dns64/tests_sh_dns64.py).

Related https://github.com/pytest-dev/pytest/issues/7777

(cherry picked from commit 7118cbed98)
2024-10-24 13:57:41 +02:00
Matthijs Mekking
a25918c9f5 chg: nil: Add inline-signing warning for upgrading to 9.20
For dynamic zones that do not set inline-signing explicitly, add a warning that the default value for inline-signing has changed. Dynamic zones that want to be able to reuse the zone (and not trigger a full resign) should
explicitly configure "inline-signing no;".

Merge branch 'matthijs-warn-if-inline-signing-not-set-for-dynamic-zone-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9646
2024-10-23 13:19:30 +00:00
Matthijs Mekking
e5c711fd43 Add inline-signing warning for upgrading to 9.20
For dynamic zones that do not set inline-signing explicitly, add a
warning that the default value for inline-signing has changed. Dynamic
zones that want to be able to reuse the zone (and not trigger a full
resign) should explicitly configure "inline-signing no;".
2024-10-23 10:34:49 +00:00
Michal Nowak
fa16eb402e [9.18] chg: test: Rewrite verify system test to pytest
Backport of MR !9249

Merge branch 'backport-mnowak/pytest_rewrite_verify-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9675
2024-10-21 13:07:12 +00:00
Michal Nowak
82712ad7fc
Rewrite verify system test to pytest
(cherry picked from commit 091968f5fa)
2024-10-21 14:11:46 +02:00
Michal Nowak
59e20811fc [9.18] fix: doc: All footnotes need to have unique name and be referenced
With Sphinx 8.1.0, footnotes can't stand on their own and have to be
referenced from somewhere, otherwise build fails, e.g.:

    doc/dnssec-guide/signing.rst:1470: WARNING: Footnote [#] is not referenced. [ref.footnote]

Backport of MR !9663

Merge branch 'backport-mnowak/fix-dnssec-guide-footnote-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9671
2024-10-21 09:15:55 +00:00
Michal Nowak
eb9061e7f1
All footnotes need to have unique name and be referenced
With Sphinx 8.1.0, footnotes can't stand on their own and have to be
referenced from somewhere, otherwise build fails, e.g.:

    doc/dnssec-guide/signing.rst:1470: WARNING: Footnote [#] is not referenced. [ref.footnote]

(cherry picked from commit bdf8859e2d)
2024-10-21 11:02:27 +02:00
Mark Andrews
002141af2e [9.18] fix: usr: Restore values when dig prints command line
Options of the form `[+-]option=<value>` failed to display the value on the printed command line. This has been fixed.

Closes #4993

Backport of MR !9653

Merge branch 'backport-4993-dig-restore-command-line-values-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9667
2024-10-18 02:59:14 +00:00
Mark Andrews
a05508b0c4 Correctly get final token
(cherry picked from commit df5b4ba894)
2024-10-18 02:10:39 +00:00
Mark Andrews
b22134fd74 Restore seperator values
strtok_r is destructive.  Restore the seperators so that the command
line can be properly displayed.

(cherry picked from commit 609d96aa12)
2024-10-18 02:10:38 +00:00
Mark Andrews
7bb817d1b6 [9.18] fix: dev: Remove unused <openssl/{hmac,engine}.h> headers from OpenSSL shims
The <openssl/{hmac,engine}.h> headers were unused and including the
<openssl/engine.h> header might cause build failure when OpenSSL
doesn't have Engines support enabled.

See https://fedoraproject.org/wiki/Changes/OpensslDeprecateEngine

Backport of MR !9593

Merge branch 'backport-bind-9.20-openssl-engine-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9645
2024-10-18 01:36:41 +00:00
Petr Menšík
e5ffa52c6d Remove unused <openssl/{hmac,engine}.h> headers from OpenSSL shims
The <openssl/{hmac,engine}.h> headers were unused and including the
<openssl/engine.h> header might cause build failure when OpenSSL
doesn't have Engines support enabled.

See https://fedoraproject.org/wiki/Changes/OpensslDeprecateEngine

(cherry picked from commit 75a50925f7)
2024-10-18 01:29:27 +00:00
Nicki Křížek
c5b6eb3619 [9.18] chg: test: Disable too-many/too-few pylint checks
Backport of MR !9626

Merge branch 'backport-nicki/pylint-disable-too-few-too-many-checks-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9657
2024-10-17 11:56:13 +00:00
Nicki Křížek
9ca6fcc4d5 Disable too-many/too-few pylint checks
Enforcing pylint standards and default for our test code seems
counter-productive. Since most of the newly added code are tests or is
test-related, encountering these checks rarely make us refactor the code
in other ways and we just disable these checks individually. Code that
is too complex or convoluted will be pointed out in reviews anyways.

(cherry picked from commit 7639c58c48)
2024-10-17 13:42:08 +02:00
Michal Nowak
68ac0194ee [9.18] chg: doc: Read the Docs should always source requirements.txt from "main"
HTTP URI of doc/arm/requirements.txt on the "main" branch is the source
of PyPI packages in CI base image. To make it consistent in all
maintained CI branches on RTD, use the HTTP URI as well.

This change is non-material on the "main" branch, but ensures the right
packages on stable branches where for RTD we currently leverage outdated
packages because we failed to update doc/arm/requirements.txt there.

Backport of MR !9630

Merge branch 'backport-mnowak/rtd-always-resource-requirements-txt-from-main-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9661
2024-10-17 11:23:12 +00:00
Michal Nowak
d189b8a432
Drop doc/arm/requirements.txt
doc/arm/requirements.txt is maintained only on the "main" branch.
2024-10-17 12:59:31 +02:00
Michal Nowak
03d068d168 Read the Docs should always source requirements.txt from "main"
HTTP URI of doc/arm/requirements.txt on the "main" branch is the source
of PyPI packages in CI base image. To make it consistent in all
maintained CI branches on RTD, use the HTTP URI as well.

This change is non-material on the "main" branch, but ensures the right
packages on stable branches where for RTD we currently leverage outdated
packages because we failed to update doc/arm/requirements.txt there.

(cherry picked from commit 8fb6115492)
2024-10-17 10:50:25 +00:00
Michal Nowak
9ddfe87a76 [9.18] chg: doc: Update Sphinx, docutils, and sphinx_rtd_theme packages
Backport of MR !9652

Merge branch 'backport-mnowak/doc-tools-october-2024-updates-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9659
2024-10-17 10:28:04 +00:00
Michal Nowak
2405885b05 Add RFCs referenced by footnotes
With Sphinx 8.1.0, footnotes can't stand on their own and have
referenced from somewhere:

    /builds/isc-projects/bind9/doc/arm/general.rst:439: WARNING: Footnote [#] is not referenced. [ref.footnote]
    /builds/isc-projects/bind9/doc/arm/general.rst:441: WARNING: Footnote [#] is not referenced. [ref.footnote]
    /builds/isc-projects/bind9/doc/arm/general.rst:445: WARNING: Footnote [#] is not referenced. [ref.footnote]
    /builds/isc-projects/bind9/doc/arm/general.rst:457: WARNING: Footnote [#] is not referenced. [ref.footnote]

(cherry picked from commit 54410e034f)
2024-10-17 10:11:19 +00:00
Michal Nowak
992dd6f2e3 Update Sphinx, docutils, and sphinx_rtd_theme packages
(cherry picked from commit 755dd44ec5)
2024-10-17 10:11:19 +00:00
Michal Nowak
55c09a4d95 Merge tag 'v9.18.31' into bind-9.18 2024-10-16 17:54:38 +02:00
Michal Nowak
8ce45e2a72 [9.18] chg: test: Remove unused sbytes, stime, n, response, and rtime variables
Backport of MR !9617

Merge branch 'backport-mnowak/drop-unused-variables-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9644
2024-10-15 18:57:23 +00:00
Michal Nowak
5d7a32f5c4 Add vulture
(cherry picked from commit c0246ecef6)
2024-10-15 18:50:06 +00:00
Michal Nowak
03cf294d06 Remove unused sbytes, stime, n, response, and rtime variables
(cherry picked from commit 4857da271d)
2024-10-15 18:50:06 +00:00
Michal Nowak
e0ad4d5eb6 [9.18] chg: test: Rewrite tools system test to pytest
Backport of MR !9208

Merge branch 'backport-mnowak/pytest_rewrite_tools-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9642
2024-10-15 18:15:06 +00:00
Michal Nowak
eb8a56c90c Rewrite tools system test to pytest
(cherry picked from commit 3f8e59330d)
2024-10-15 17:28:42 +00:00
Michal Nowak
4bbb46ea84 [9.18] chg: doc: Update sphinx to 7.4.7
Backport of MR !9529.

Merge branch 'backport-mnowak/update-sphinx-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9639
2024-10-15 12:54:24 +00:00
Nicki Křížek
4f1e800f28
Update sphinx to 7.4.7
(cherry picked from commit 3d65fa2740)
2024-10-15 14:39:31 +02:00
Nicki Křížek
6cb3cc642b [9.18] chg: test: Support dnspython 2.7.0
Closes #4988

Backport of MR !9634

Merge branch 'backport-4988-dnspython-2.7.0-support-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9636
2024-10-15 11:09:16 +00:00
Nicki Křížek
ef508f5052 Support dnspython 2.7.0
CookieOption with new .server/.client attributes (rather than .data) was
added to dnspython. Adjust the code to use the new attributes if
available and fall back to the old code for dnspython<2.7.0
compatibility.

(cherry picked from commit 0d90b13646)
2024-10-15 10:30:02 +00:00
Mark Andrews
9d59c72798 [9.18] fix: usr: Fix NSEC3 closest encloser lookup for names with empty non-terminals
The performance improvement for finding the NSEC3 closest encloser when generating authoritative responses could cause servers to return incorrect NSEC3 records in some cases. This has been fixed.

Closes #4950

Backport of MR !9610

Merge branch 'backport-4950-bind-logs-expected-covering-nsec3-got-an-exact-match-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9632
2024-10-15 02:37:02 +00:00
Mark Andrews
74b6aa8dd2 Test that the correct NSEC3 closest encloser is returned
(cherry picked from commit b457f64d4a)
2024-10-15 11:00:03 +11:00
Mark Andrews
94f6655915 Use a binary search to find the NSEC3 closest encloser
maxlabels is the suffix length that corresponds to the latest
NXDOMAIN response.  minlabels is the suffix length that corresponds
to longest found existing name.

(cherry picked from commit 67f31c5046)
2024-10-14 23:55:13 +00:00
Matthijs Mekking
f60f153b8a [9.18] chg: usr: Harden key management when key files have become unavailabe
Prior to doing key management, BIND 9 will check if the key files on disk match the expected keys. If key files for previously observed keys have become unavailable, this will prevent the internal key manager from running.

Backport of MR !9337

Merge branch 'backport-4763-do-not-roll-if-key-files-are-missing-9.18' into 'bind-9.18'

See merge request isc-projects/bind9!9623
2024-10-14 14:29:05 +00:00
Matthijs Mekking
16fed98e20 Add new behavior to the ARM
Add text to the ARM that describes what we do in case key files have
become unavailable.

(cherry picked from commit 351c066d91)
2024-10-14 13:54:09 +00:00
Matthijs Mekking
39e4576341 Test removing DNSKEYs from other providers
In a multi-signer setup, removing DNSKEY records from the zone should
not be treated as a key that previously exists in the keyring, thus
blocking the keymgr. Add a test case to make sure.

(cherry picked from commit 5f552293d7)
2024-10-14 13:54:09 +00:00