From edd765a092339816c3d3fc97f03acf92e73e3ee2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20K=C4=99pie=C5=84?= Date: Thu, 2 Jul 2026 15:07:40 +0200 Subject: [PATCH] Move ans.py-related information to README.md COOKBOOK.md is supposed to be minimal and heavy on examples, so move the lengthy section about implementing custom ans.py servers from COOKBOOK.md to README.md. --- bin/tests/system/COOKBOOK.md | 209 ----------------------------------- bin/tests/system/README.md | 209 +++++++++++++++++++++++++++++++++++ 2 files changed, 209 insertions(+), 209 deletions(-) diff --git a/bin/tests/system/COOKBOOK.md b/bin/tests/system/COOKBOOK.md index b4f5152858..08e51a1d84 100644 --- a/bin/tests/system/COOKBOOK.md +++ b/bin/tests/system/COOKBOOK.md @@ -135,215 +135,6 @@ Notes: artifact check at teardown will fail. Most real test modules carry one. -## Write a regression reproducer - -The goal: turn "issue #NNNN" into a failing test with minimal ceremony. - -1. **Decide the server topology.** Most reproducers need one of: - - a single authoritative `named` (answer content bugs) — the skeleton - recipe above; - - a resolver plus a mock server that misbehaves (resolver bugs) — see - the mock server recipe below; - - signed zones and a validating resolver (DNSSEC bugs) — see the zone - setup recipe below. - -2. **Find the closest existing test and copy its shape.** Good exemplars: - `cyclic_glue` (resolver + python mock server), `dnssec_py` (signed zones, - validator, multiple modules sharing one server set), `nsec3` (multi-module - family), `kasp`/`rollover_*` (key management state machines). - -3. **Decide where the test lives.** If an existing directory already has the - server set you need, add a new `tests_*.py` module there; otherwise create - a new directory. Each module gets its own temporary directory, port - range, and parallel slot, so you are not entangled with the other modules. - Test functions *within* a module, however, run in file order against the - same live servers: a new test inherits whatever state the tests above it - left behind (cache contents, dynamic updates) and can disturb the tests - below it. - -4. **Write the test to fail first.** Run it against an unfixed build and - make sure it fails for the reason the issue describes — `ns*/named.run` - in the kept temporary directory is the place to verify that. Then apply - the fix and watch it pass. - - -## Mock a misbehaving server - -When a test needs a server that answers in ways named never would (bogus -glue, truncation, dropped queries, malformed records), add an `ansN` -subdirectory containing an `ans.py` script based on `isctest.asyncserver`. -The runner starts it automatically on 10.53.0.N, logging to `ans.run`. - -Implementing a custom `ansN` server happens in two phases: - - - define all static DNS data that the server needs to serve (if any) in `*.db` - files, like you would for a regular `named` instance, - - - implement any non-standard behavior (modifying zone-based responses or - generating responses from scratch) by defining a response handler class, - scoping it to the QNAMEs/QTYPEs/domains it owns, and installing it into an - `AsyncDnsServer`. - -Most importantly, avoid the temptation to define all DNS responses that a given -`ansN` server needs to serve using just dnspython APIs; zone files are much -easier to follow for static DNS data. Splitting up static DNS data and custom -behavior also makes it easier to follow the idea behind each test. - -The most commonly subclassed handler classes are (ordered by descending -specificity): - - - `QnameQtypeHandler` - - `QnameHandler` - - `DomainHandler` - -These handler classes require certain properties (e.g. `qnames`, `qtypes`, -`domains`) to be defined by their subclasses. These properties define the set -of queries that a given handler should be used for. Please see -`isctest/asyncserver.py` for up-to-date information on available handler classes -and existing `ans.py` files for how they can be used in practice. Consult the -log files (`ans.run`) in case a query is not matched by its intended handler. - -**NOTE:** For readability (of both code and logs), defining separate handler -classes for distinct queries is strongly preferred over using a single handler -containing an `if`/`elif`/`else` chain. - -**NOTE:** If you find yourself implementing an `__init__()` method in your -handler subclass, it often indicates that you're approaching the problem at hand -from the wrong side; contact QA for guidance in such a case. - -When a query is matched to a handler, the latter is expected to yield a response -action through its `get_responses()` method, an async generator that inspects -the query context and decides how the server should react: - -```python -from collections.abc import AsyncGenerator - -import dns.flags - -from isctest.asyncserver import ( - AsyncDnsServer, - DnsResponseSend, - DomainHandler, - QueryContext, - ResponseAction, -) - - -class TruncateHandler(DomainHandler): - """Answer everything under broken.example. with TC=1.""" - - domains = ["broken.example."] - - async def get_responses( - self, qctx: QueryContext - ) -> AsyncGenerator[ResponseAction, None]: - qctx.response.flags |= dns.flags.TC - yield DnsResponseSend(qctx.response) - - -def main() -> None: - server = AsyncDnsServer() - server.install_response_handler(TruncateHandler()) - server.run() - - -if __name__ == "__main__": - main() -``` - -The available response actions are `DnsResponseSend` (optionally with a -`delay`), `ResponseDrop` (don't answer at all), `BytesResponseSend` (raw -bytes, for malformed packets) and `CloseConnection` (TCP). Queries that no -handler matches are answered from zone data — `AsyncDnsServer` loads every -`*.db` zone file found in the `ansN` directory at startup — or with the -server's default rcode (REFUSED unless configured otherwise). - -**NOTE:** For returning static responses, subclassing `StaticResponseHandler` is -strongly recommended instead of implementing the `get_responses()` generator -manually; see `resolver/ans3/ans.py` for practical examples. - -**NOTE:** Calling `yield` does **NOT** make `get_responses()` return! This is -by design: `get_responses()` can yield multiple DNS messages in response to a -single query, so that it can also handle AXFR/IXFR queries, among others. Be -careful not to unintentionally cause multiple DNS messages to be returned for a -single query. If your handler's `get_responses()` method contains multiple -`yield` statements, it might be a sign that it needs to be refactored into -multiple separate handlers. - -If multiple `ansN` instances used in a given system test need to share common -logic, extract that logic into a `_ans.py` module in the system test -directory. See the `qmin` system test for a practical example. - -If multiple system tests would benefit from sharing some common logic, consider -submitting a merge request adding that logic to `isctest/asyncserver.py` itself. - -To the extent possible, try to keep each `ans.py` file limited in length and -scope. Look at existing `ans.py` files to see what is meant by that. If the -response generation logic required for reproducing a given bug is particularly -complex, consider dedicating the entire `ans.py` file just to that logic instead -of appending it to an existing one; `ansN` instances are cheap to spawn and run -compared to regular `named` instances. If the number of `ansN` instances used -in a given system test is becoming unwieldy, it usually indicates the need to -start adding/moving code to a new system test directory. - -In some rare cases, it may be useful to reuse a common set of `nsN` server -instances to reproduce a whole class of related issues, triggering which relies -on some non-standard behavior and therefore needs a custom `ansN` server to be -implemented. If the logic necessary for reproducing each of these issues is -complex and the amount of those issues makes it impractical to add a separate -`ansN` server for each issue (as recommended in the previous paragraph), it is -acceptable to split up the test logic for each issue into separate `ans_*.py` -modules inside a single `ansN` directory and reduce `ans.py` itself to a loader -that imports and installs handlers defined in those separate modules: - -```python -from mytest.ans1 import ans_some_bug, ans_some_other_bug -from isctest.asyncserver import AsyncDnsServer - - -def main() -> None: - server = AsyncDnsServer() - server.install_response_handler(ans_some_bug.SomeBugHandler()) - server.install_response_handler(ans_some_other_bug.SomeOtherBugHandler()) - server.run() - - -if __name__ == "__main__": - main() -``` - -However, in such a case it is particularly important to ensure consistency -between the names of all the Python files related to a given issue - otherwise, -chaos ensues. Furthermore, avoid using cryptic file names (e.g. numeric bug -identifiers). The recommended naming scheme is: - -``` -mytest/ -├── ans1 -│   ├── ans.py -│   ├── ans_some_bug.py -│   └── ans_some_other_bug.py -├── ns2 -│   └── ... -├── tests_some_bug.py -└── tests_some_other_bug.py -``` - -To point a resolver at the mock, delegate to it from the test's root zone -(served by ns1) or list it as a forwarder; `cyclic_glue` shows the -delegation pattern end to end. - -The existing mock servers are the best reference. To find them, grep for -what you're about to use: -`git grep -l isctest.asyncserver -- '*/ans*/ans.py'` lists every python -mock, and a grep for the base class -(`DomainHandler`, `QnameHandler`, `ConnectionHandler`) or the response -action (`ResponseDrop`, `BytesResponseSend`, ...) you need usually turns -up a test already doing something similar. The full toolbox lives in -`isctest/asyncserver.py` (query matching, TCP connection handling, TSIG -keyrings). - - ## Set up zones in bootstrap() A module-level `bootstrap()` function runs before the config templates are diff --git a/bin/tests/system/README.md b/bin/tests/system/README.md index 171af58e9c..b442d69572 100644 --- a/bin/tests/system/README.md +++ b/bin/tests/system/README.md @@ -396,6 +396,215 @@ comes to naming files and test functions). New system test directories are discovered automatically; no registration in the build system is needed. +### Writing a regression reproducer + +The goal: turn "issue #NNNN" into a failing test with minimal ceremony. + +1. **Decide the server topology.** Most reproducers need one of: + - a single authoritative `named` (answer content bugs) — the skeleton + recipe above; + - a resolver plus a mock server that misbehaves (resolver bugs) — see + the mock server recipe below; + - signed zones and a validating resolver (DNSSEC bugs) — see the zone + setup recipe below. + +2. **Find the closest existing test and copy its shape.** Good exemplars: + `cyclic_glue` (resolver + python mock server), `dnssec_py` (signed zones, + validator, multiple modules sharing one server set), `nsec3` (multi-module + family), `kasp`/`rollover_*` (key management state machines). + +3. **Decide where the test lives.** If an existing directory already has the + server set you need, add a new `tests_*.py` module there; otherwise create + a new directory. Each module gets its own temporary directory, port + range, and parallel slot, so you are not entangled with the other modules. + Test functions *within* a module, however, run in file order against the + same live servers: a new test inherits whatever state the tests above it + left behind (cache contents, dynamic updates) and can disturb the tests + below it. + +4. **Write the test to fail first.** Run it against an unfixed build and + make sure it fails for the reason the issue describes — `ns*/named.run` + in the kept temporary directory is the place to verify that. Then apply + the fix and watch it pass. + + +### Mock a misbehaving server + +When a test needs a server that answers in ways named never would (bogus +glue, truncation, dropped queries, malformed records), add an `ansN` +subdirectory containing an `ans.py` script based on `isctest.asyncserver`. +The runner starts it automatically on 10.53.0.N, logging to `ans.run`. + +Implementing a custom `ansN` server happens in two phases: + + - define all static DNS data that the server needs to serve (if any) in `*.db` + files, like you would for a regular `named` instance, + + - implement any non-standard behavior (modifying zone-based responses or + generating responses from scratch) by defining a response handler class, + scoping it to the QNAMEs/QTYPEs/domains it owns, and installing it into an + `AsyncDnsServer`. + +Most importantly, avoid the temptation to define all DNS responses that a given +`ansN` server needs to serve using just dnspython APIs; zone files are much +easier to follow for static DNS data. Splitting up static DNS data and custom +behavior also makes it easier to follow the idea behind each test. + +The most commonly subclassed handler classes are (ordered by descending +specificity): + + - `QnameQtypeHandler` + - `QnameHandler` + - `DomainHandler` + +These handler classes require certain properties (e.g. `qnames`, `qtypes`, +`domains`) to be defined by their subclasses. These properties define the set +of queries that a given handler should be used for. Please see +`isctest/asyncserver.py` for up-to-date information on available handler classes +and existing `ans.py` files for how they can be used in practice. Consult the +log files (`ans.run`) in case a query is not matched by its intended handler. + +**NOTE:** For readability (of both code and logs), defining separate handler +classes for distinct queries is strongly preferred over using a single handler +containing an `if`/`elif`/`else` chain. + +**NOTE:** If you find yourself implementing an `__init__()` method in your +handler subclass, it often indicates that you're approaching the problem at hand +from the wrong side; contact QA for guidance in such a case. + +When a query is matched to a handler, the latter is expected to yield a response +action through its `get_responses()` method, an async generator that inspects +the query context and decides how the server should react: + +```python +from collections.abc import AsyncGenerator + +import dns.flags + +from isctest.asyncserver import ( + AsyncDnsServer, + DnsResponseSend, + DomainHandler, + QueryContext, + ResponseAction, +) + + +class TruncateHandler(DomainHandler): + """Answer everything under broken.example. with TC=1.""" + + domains = ["broken.example."] + + async def get_responses( + self, qctx: QueryContext + ) -> AsyncGenerator[ResponseAction, None]: + qctx.response.flags |= dns.flags.TC + yield DnsResponseSend(qctx.response) + + +def main() -> None: + server = AsyncDnsServer() + server.install_response_handler(TruncateHandler()) + server.run() + + +if __name__ == "__main__": + main() +``` + +The available response actions are `DnsResponseSend` (optionally with a +`delay`), `ResponseDrop` (don't answer at all), `BytesResponseSend` (raw +bytes, for malformed packets) and `CloseConnection` (TCP). Queries that no +handler matches are answered from zone data — `AsyncDnsServer` loads every +`*.db` zone file found in the `ansN` directory at startup — or with the +server's default rcode (REFUSED unless configured otherwise). + +**NOTE:** For returning static responses, subclassing `StaticResponseHandler` is +strongly recommended instead of implementing the `get_responses()` generator +manually; see `resolver/ans3/ans.py` for practical examples. + +**NOTE:** Calling `yield` does **NOT** make `get_responses()` return! This is +by design: `get_responses()` can yield multiple DNS messages in response to a +single query, so that it can also handle AXFR/IXFR queries, among others. Be +careful not to unintentionally cause multiple DNS messages to be returned for a +single query. If your handler's `get_responses()` method contains multiple +`yield` statements, it might be a sign that it needs to be refactored into +multiple separate handlers. + +If multiple `ansN` instances used in a given system test need to share common +logic, extract that logic into a `_ans.py` module in the system test +directory. See the `qmin` system test for a practical example. + +If multiple system tests would benefit from sharing some common logic, consider +submitting a merge request adding that logic to `isctest/asyncserver.py` itself. + +To the extent possible, try to keep each `ans.py` file limited in length and +scope. Look at existing `ans.py` files to see what is meant by that. If the +response generation logic required for reproducing a given bug is particularly +complex, consider dedicating the entire `ans.py` file just to that logic instead +of appending it to an existing one; `ansN` instances are cheap to spawn and run +compared to regular `named` instances. If the number of `ansN` instances used +in a given system test is becoming unwieldy, it usually indicates the need to +start adding/moving code to a new system test directory. + +In some rare cases, it may be useful to reuse a common set of `nsN` server +instances to reproduce a whole class of related issues, triggering which relies +on some non-standard behavior and therefore needs a custom `ansN` server to be +implemented. If the logic necessary for reproducing each of these issues is +complex and the amount of those issues makes it impractical to add a separate +`ansN` server for each issue (as recommended in the previous paragraph), it is +acceptable to split up the test logic for each issue into separate `ans_*.py` +modules inside a single `ansN` directory and reduce `ans.py` itself to a loader +that imports and installs handlers defined in those separate modules: + +```python +from mytest.ans1 import ans_some_bug, ans_some_other_bug +from isctest.asyncserver import AsyncDnsServer + + +def main() -> None: + server = AsyncDnsServer() + server.install_response_handler(ans_some_bug.SomeBugHandler()) + server.install_response_handler(ans_some_other_bug.SomeOtherBugHandler()) + server.run() + + +if __name__ == "__main__": + main() +``` + +However, in such a case it is particularly important to ensure consistency +between the names of all the Python files related to a given issue - otherwise, +chaos ensues. Furthermore, avoid using cryptic file names (e.g. numeric bug +identifiers). The recommended naming scheme is: + +``` +mytest/ +├── ans1 +│   ├── ans.py +│   ├── ans_some_bug.py +│   └── ans_some_other_bug.py +├── ns2 +│   └── ... +├── tests_some_bug.py +└── tests_some_other_bug.py +``` + +To point a resolver at the mock, delegate to it from the test's root zone +(served by ns1) or list it as a forwarder; `cyclic_glue` shows the +delegation pattern end to end. + +The existing mock servers are the best reference. To find them, grep for +what you're about to use: +`git grep -l isctest.asyncserver -- '*/ans*/ans.py'` lists every python +mock, and a grep for the base class +(`DomainHandler`, `QnameHandler`, `ConnectionHandler`) or the response +action (`ResponseDrop`, `BytesResponseSend`, ...) you need usually turns +up a test already doing something similar. The full toolbox lives in +`isctest/asyncserver.py` (query matching, TCP connection handling, TSIG +keyrings). + + ## Nameservers As noted earlier, a system test will involve a number of nameservers. These