From 948cadade2914545153746aab9a6b5ce8e48dc85 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nicki=20K=C5=99=C3=AD=C5=BEek?= Date: Fri, 21 Nov 2025 14:35:57 +0100 Subject: [PATCH 1/2] Set stricter respdiff:recent-named threshold This job is testing the current BIND implementation against the latest released version. Unless there has been a behaviour change, there should be no difference. In practice, there is a small number of differences caused by upstream discrepencies. Some of those cause "upstream unstable" answers which are excluded from the results, but statistically, some of those will end up being detected as differences on the resolver under test. Currently, there seems to be about 300 upstream unstable answers with typically around 50-60 differences. Setting the threshold to 0.1 should be stable enough to pass if there are no changes, yet sensitive enough to detect even fairly small changes to behaviour. (cherry picked from commit 92ae68764954b3606a899f5f3ecd16846cda7324) --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ef95cd16c2..99afebdb74 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -2151,7 +2151,7 @@ respdiff:recent-named: variables: CC: gcc CFLAGS: "${CFLAGS_COMMON} -Og -DISC_TRACK_PTHREADS_OBJECTS" - MAX_DISAGREEMENTS_PERCENTAGE: "0.3" + MAX_DISAGREEMENTS_PERCENTAGE: "0.1" # Performance tests From 4a6f9c5900adc49a3216a9b8bf1ceda3948c3a51 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nicki=20K=C5=99=C3=AD=C5=BEek?= Date: Fri, 21 Nov 2025 15:05:36 +0100 Subject: [PATCH 2/2] Increase the threshold for respdiff-third-party There are multiple reasons for the increased amount of differences we've been seeing lately and for the raise of the threshold: 1. Recent hardening against cache poisoning (CVE-2025-40778) have uncovered a few edge cases where the domain can't be properly resolved with the new protections in place, but those are issues with upstream configuration and DNS setup. 2. The same hardening magnified some behaviour differences between 9.21 and older versions. Some misconfigured domains, which can be resolved with BIND 9.20 and older are no longer resolvable in 9.21+. This can be again attributed to upstream DNS misconfiguration. See #5649. 3. A change in the respdiff CI job to include timeouts in the comparison, or rather, increasing the timeouts to resolve the previously timed out queries, which are typically failures. With the previous job configuration, those were omitted from comparison, because they were timeouts. Now, there should be no timeouts, but there is a slight increase in the amount of differences for the threshold evaluation. (cherry picked from commit bcc4369b0bf243433ca5334cdce3982a15ce4027) --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 99afebdb74..da4d59d179 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -2122,7 +2122,7 @@ respdiff-third-party: variables: CC: gcc CFLAGS: "${CFLAGS_COMMON} -Og" - MAX_DISAGREEMENTS_PERCENTAGE: "0.3" + MAX_DISAGREEMENTS_PERCENTAGE: "0.4" script: - bash respdiff.sh -s third_party -q "${PWD}/100k_mixed.txt" -c 1 -w "${PWD}/rspworkdir" "${CI_PROJECT_DIR}" - cd ../.. && make clean >/dev/null 2>&1