qp-tries allocate their nodes (twigs) in chunks to reduce allocator
pressure and improve memory locality. The choice of chunk size presents
a tradeoff: larger chunks benefit qp-tries with many values (as seen
in large zones and resolvers) but waste memory in smaller use cases.
Previously, our fixed chunk size of 2^10 twigs meant that even an
empty qp-trie would consume 12KB of memory, while reducing this size
would negatively impact resolver performance.
This commit implements an adaptive chunking strategy that:
- Tracks the size of the most recently allocated chunk.
- Doubles the chunk size for each new allocation until reaching a
predefined maximum.
This approach effectively balances memory efficiency for small tries
while maintaining the performance benefits of larger chunk sizes for
bigger data structures.
This commit also splits the callback freeing qpmultis into two
phases, one that frees the underlying qptree, and one that reclaims
the qpmulti memory. In order to prevent races between the qpmulti
destructor and chunk garbage collection jobs, the second phase is
protected by reference counting.
(cherry picked from commit 70b1777d8a)
When the zone is configured with a CNAME override policy, the EDE code
is not added as expected. Add a test case based on the issue in GitLab
(#5342).
When the zone contains a wildcard CNAME, the EDE code is not added as
expected. Also add a test case for this.
(cherry picked from commit 4ef00f97d6)
Rather than using the dnspython's facilities and defaults to create the
queries, use the isctest.query.create function in all the cases that
don't require special handling to have consistent defaults.
(cherry picked from commit 64143ea077)
Make the query helper function more universal and reusable across our
system tests -- default to using EDNS and sending AD=1.
(cherry picked from commit 989e64b9b0)
Use a common function to count the number of RRs in any section of the
DNS message. For the ADDITIONAL section, stick with the dnspython
convention of not including OPT and TSIG.
(cherry picked from commit efd60348b9)
added some helper functions in isctest to reduce code repetition
in dnssec-related tests:
- isctest.check.adflag() - checks that a response contains AD=1
- isctest.check.noadflag() - checks that a response contains AD=0
- isctest.check.rdflag() - checks that a response contains RD=1
- isctest.check.nordflag() - checks that a response contains RD=0
- isctest.check.answer_count_eq() - checks the answer count is correct
- isctest.check.additional_count_eq() - same for authority count
- isctest.check.authority_count_eq() - same for additional count
- isctest.check.same_data() - check that two message have the
same rcode and data
- isctest.check.same_answer() - check that two message have the same
rcode and answer
- isctest.dnssec.msg() - a wrapper for dns.message.make_query() that
creates a query message similar to dig +dnssec:
use_edns=True, want_dnssec=True,
and flags are set to (RD|AD) by default, but
options exist to disable AD or enable CD.
(to generate non-DNSSEC queries, use
message.make_query() directly.)
(cherry picked from commit b69097f139)
MR !10238 added key collision detection in the ksr system test but it
was flawed because for every "collide" in the output we also log
"Generating an new key" and for each "Generating" we add the counter
by one, nullifying the subtract by one.
Use regular expressions to search in the output and make the string
expression more strict.
(cherry picked from commit abdb9a1334)
Add missing type hints in the tests_nsec3.py module. Tweak the syntax
used for type hints for better consistency with other Python code in
bin/tests/system/.
(cherry picked from commit adb931f700)
This should prevent the case where are are unlucky enough that static
values hash 'just right' for the test to pass, but only accidentally.
(cherry picked from commit 46781845ea)
Currently this test is limited only to auth because currently BIND
resolver does not send DS proof of nonexistence for RD=0 queries.
(cherry picked from commit 548632b18a)
Simplistic test. Ignores the possibility of DNAME chain going through
multiple zones and/or wildcard expansions.
(cherry picked from commit 73e4201331)
Simplistic test. Ignores the possibility of CNAME chain going through
multiple zones and/or wildcard expansions.
(cherry picked from commit d0e413dd57)
The test actually needs just two servers - auth and resolver. The rest
was not needed and made test setup only slower and harder to debug.
(cherry picked from commit ac58b58002)
Basic sanity checks - limited to responses from a single zone:
- NSEC3 type cannot be present in type bitmap:
By definition, the type bitmap describes state of the unhashed name
but NSEC3 RR is present at a different owner name. RFC 7129 section 5
- NSEC3 owner names cannot be duplicated:
Unless the response crosses zone boundary, parent zone has insecure
delegation for child, but child is signed ... don't do that.
- All parameters are consistent across all RRs present in answer:
RFC 5155 section 7.2, last paragraph - at least when we don't cross
zone boundary.
(cherry picked from commit cfaf5c997f)
Untangling individual cases allows for clearer documentation and makes
it easier to build similar but slightly different test cases. Wildcard
NODATA answer was added.
(cherry picked from commit 9ca2077274)
As a side-effect, we now have set of all existing names in a zone with a
test, too. These parts should be shared with new NSEC tests.
(cherry picked from commit f0592de608)
Side-effect of importing from isctest.hypothesis first is a version
check and clean Pytest skip if version is too old.
(cherry picked from commit 9cea2af25c)
Test all combinations of wildcard, ENT, DNAME, NS, and ordinary
TXT records.
Test zone and expected outputs are generated by another script which
encodes node content into node name. This encoding removes 'node
content' level of indirection and thus enables simpler implementation of
same logic which needs to be in ZoneAnalyzer itself.
For humans the generated zone file also lists expected 'categories' a
name belongs to as dot-separated list on right hand side of a generated
RR.
(cherry picked from commit 42b60a3819)
I've considered writing hypothesis test for this but I would have to
reimplement the same thing, which would probably have the same logic
bugs, so I will leave it as an exercise for someone else.
(cherry picked from commit cad48e56ab)
dns.name all over the place does not make it easier to read the code at
all, and I'm going to add lot more code here.
(cherry picked from commit 3fb6b990af)
Check the correctness of NSEC3 hash generation by generating random
combinations of name, salt, and iterations and comparing the outputs
of the nsec3hash tool against the dnspython nsec3_hash function
for the same inputs.
(cherry picked from commit e263df8848)
For any given NSEC3 signed zone, when doing queries for non-existent
names, the response must contain:
- NSEC3 RR that matches the closest encloser,
- NSEC3 RR that covers the next closer name,
- NSEC3 RR that covers the wildcard.
(cherry picked from commit 955e3ccf3e)
When the tests-connreset.py module was initially implemented in commit
5c17919019, the dispatch code did not
properly apply the idle timeout to TCP connections. This allowed the
check in that test module to reset the TCP connection after 5 seconds as
named did not attempt to tear the connection down earlier than that.
However, as the dispatch code was improved, the idle timeout started
being enforced for TCP dispatches; the exact value it is set to in the
current code depends on a given server's SRTT, but it defaults to about
1.2 seconds for responsive servers. This means that the code paths
triggered by the "dispatch" system test are now different than the ones
it was originally supposed to trigger because it is now named itself
that shuts the TCP connection down cleanly before the ans3 server gets a
chance to reset it.
Account for the above by lowering the amount of time after which the
ans3 server in the "dispatch" system test resets TCP connections to just
1 second, so that the test actually does what its name implies.
(cherry picked from commit 48e705d738)
Replace the custom DNS server used in the "dispatch" system test with
new code based on the isctest.asyncserver module.
(cherry picked from commit 316b7d5590)
Add a TCP connection handler, ConnectionReset, which enables closing TCP
connections without emptying the client socket buffer, causing the
kernel to send an RST segment to the client. This relies on a horrible
asyncio hack that can break at any point in the future due to abusing
implementation details in the Python Standard Library. Despite the eye
bleeding this code may cause, the approach it takes was still deemed
preferable to implementing an asyncio transport from scratch just to
enable triggering connection resets.
(cherry picked from commit e407888507)
Add a new abstract class, ConnectionHandler, instances of which can be
installed on AsyncDnsServer to manipulate TCP connections upon
accepting.
(cherry picked from commit b4d53e7287)
In response to client queries, AsyncDnsServer users can currently only
make the server either send a reply or silently ignore the query. In
the case of TCP queries, neither of these actions causes the client's
connection to be closed - the onus of doing that is on the client.
However, in some cases the server may be required to close the
connection on its own, so AsyncDnsServer users need to have some way of
requesting such an action.
Add a new ResponseAction subclass, ResponseDropAndCloseConnection, which
enables AsyncDnsServer users to conveniently request TCP connections to
be closed. Instead of returning the response to send,
ResponseDropAndCloseConnection raises a custom exception that
AsyncDnsServer._handle_tcp() handles accordingly.
(cherry picked from commit 06b0800df8)
The original `ans.pl` server was based on a copy of the one in
`fetchlimit`, so there are some changes:
- The server now only responds with A replies (which is the only thing
needed).
- The incrementing of the IP address goes beyond the least significant
octet (so, after 192.0.2.255 it will yield 192.0.3.0).
(cherry picked from commit ec5729bee3)
The LSP server (using clangd) was always complaining about:
Suspicious string literal, probably missing a comma
for the two Local IPv6 Unicast Addresses strings that spanned
across multiple lines. Disable clang-format for these two lines.
(cherry picked from commit 6b7c99027d)
Replace the custom DNS server used in the "fetchlimit" system test
with new code based on the isctest.asyncserver module.
(cherry picked from commit 9ffc833919)
Add \007.no-apex-covering as an owner name so that the cache does
not get primed with a parent NSEC RRset to test the case where
dns_qp_lookup returns ISC_R_NOTFOUND.
(cherry picked from commit df04924209)
The kasp test cases assume that keymgr operations on the zone under test
have been completed before the test is executed. These are typically
quite fast, but the logs need to be explicitly checked for the messages,
otherwise there's a possibility of race conditions causing the
kasp/rollover tests to become unstable.
Call the wait function in all the kasp/rollover tests where it is
expected (which is generally in each test, unless we're dealing with
unsigned zones).
(cherry picked from commit 467b826162)
Many of our test cases only use a single NamedInstance from the
`servers` fixture. Introduce `nsX` helper fixtures to simplify these
tests and reduce boilterplate code further.
Specifically, the test no longer has to either define its own variable
to extract a single server from the list, or use the longer
servers["nsX"] syntax. While this may seem minor, the amount of times it
is repeated across the tests justifies the change. It also promotes
using more explicit server identification, i.e. `nsX`, rather than
generic `server`. This also improves the clarity of the tests and may be
helpful in traceback during debugging as well.
(cherry picked from commit fe55342916)
