Created isc_refcount_decrement_expect macro to test conditionally
the return value to ensure it is in expected range. Converted
unchecked isc_refcount_decrement to use isc_refcount_decrement_expect.
Converted INSIST(isc_refcount_decrement()...) to isc_refcount_decrement_expect.
There were several problems with rbt hashtable implementation:
1. Our internal hashing function returns uint64_t value, but it was
silently truncated to unsigned int in dns_name_hash() and
dns_name_fullhash() functions. As the SipHash 2-4 higher bits are
more random, we need to use the upper half of the return value.
2. The hashtable implementation in rbt.c was using modulo to pick the
slot number for the hash table. This has several problems because
modulo is: a) slow, b) oblivious to patterns in the input data. This
could lead to very uneven distribution of the hashed data in the
hashtable. Combined with the single-linked lists we use, it could
really hog-down the lookup and removal of the nodes from the rbt
tree[a]. The Fibonacci Hashing is much better fit for the hashtable
function here. For longer description, read "Fibonacci Hashing: The
Optimization that the World Forgot"[b] or just look at the Linux
kernel. Also this will make Diego very happy :).
3. The hashtable would rehash every time the number of nodes in the rbt
tree would exceed 3 * (hashtable size). The overcommit will make the
uneven distribution in the hashtable even worse, but the main problem
lies in the rehashing - every time the database grows beyond the
limit, each subsequent rehashing will be much slower. The mitigation
here is letting the rbt know how big the cache can grown and
pre-allocate the hashtable to be big enough to actually never need to
rehash. This will consume more memory at the start, but since the
size of the hashtable is capped to `1 << 32` (e.g. 4 mio entries), it
will only consume maximum of 32GB of memory for hashtable in the
worst case (and max-cache-size would need to be set to more than
4TB). Calling the dns_db_adjusthashsize() will also cap the maximum
size of the hashtable to the pre-computed number of bits, so it won't
try to consume more gigabytes of memory than available for the
database.
FIXME: What is the average size of the rbt node that gets hashed? I
chose the pagesize (4k) as initial value to precompute the size of
the hashtable, but the value is based on feeling and not any real
data.
For future work, there are more places where we use result of the hash
value modulo some small number and that would benefit from Fibonacci
Hashing to get better distribution.
Notes:
a. A doubly linked list should be used here to speedup the removal of
the entries from the hashtable.
b. https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/
Both clang-tidy and uncrustify chokes on statement like this:
for (...)
if (...)
break;
This commit uses a very simple semantic patch (below) to add braces around such
statements.
Semantic patch used:
@@
statement S;
expression E;
@@
while (...)
- if (E) S
+ { if (E) { S } }
@@
statement S;
expression E;
@@
for (...;...;...)
- if (E) S
+ { if (E) { S } }
@@
statement S;
expression E;
@@
if (...)
- if (E) S
+ { if (E) { S } }
Also disable the semantic patch as the code needs tweaks here and there because
some destroy functions might not destroy the object and return early if the
object is still in use.
isc_event_allocate() calls isc_mem_get() to allocate the event structure. As
isc_mem_get() cannot fail softly (e.g. it never returns NULL), the
isc_event_allocate() cannot return NULL, hence we remove the (ret == NULL)
handling blocks using the semantic patch from the previous commit.
The json-c have previously leaked into the global namespace leading
to forced -I<include_path> for every compilation unit using isc/xml.h
header. This MR fixes the usage making the caller object opaque.
The libxml2 have previously leaked into the global namespace leading
to forced -I<include_path> for every compilation unit using isc/xml.h
header. This MR fixes the usage making the caller object opaque.
Since 2008, the cleaning-interval timer has been documented as
"effectively obsolete" and disabled in the default configuration with
a comment saying "now meaningless".
This change deletes all the code that implements the cleaning-interval
timer, except for the config parser in whcih it is now explicitly
marked as obsolete.
I have verified (using the deletelru and deletettl cache stats) that
named still cleans the cache after this change.
Replace dns_fixedname_init() calls followed by dns_fixedname_name()
calls with calls to dns_fixedname_initname() where it is possible
without affecting current behavior and/or performance.
This patch was mostly prepared using Coccinelle and the following
semantic patch:
@@
expression fixedname, name;
@@
- dns_fixedname_init(&fixedname);
...
- name = dns_fixedname_name(&fixedname);
+ name = dns_fixedname_initname(&fixedname);
The resulting set of changes was then manually reviewed to exclude false
positives and apply minor tweaks.
It is likely that more occurrences of this pattern can be refactored in
an identical way. This commit only takes care of the low-hanging fruit.
allows named to provide stale cached answers when
the authoritative server is under attack.
See max-stale-ttl, stale-answer-enable,
stale-answer-ttl. [RT #44790]
3882. [func] By default, negative trust anchors will be tested
periodically to see whether data below them can be
validated, and if so, they will be allowed to
expire early. The "rndc nta -force" option
overrides this behvaior. The default NTA lifetime
and the recheck frequency can be configured by the
"nta-lifetime" and "nta-recheck" options. [RT #36146]
3524. [func] Added an alternate statistics channel in JSON format,
when the server is built with the json-c library:
http://[address]:[port]/json. [RT #32630]
3506. [func] When setting "max-cache-size" and "max-acache-size",
the keyword "unlimited" is no longer defined as equal
to 4 gigabytes (except on 32-bit platforms); it
means literally unlimited. [RT #32358]
3505. [bug] When setting "max-cache-size" and "max-acache-size",
larger values than 4 gigabytes could not be set
explicitly, though larger sizes were available
when setting cache size to 0. This has been
corrected; the full range is now available.
[RT #32358]
new stats that were added for 9.10 (changes 3319-3326) were not
all updated when the new statistics schema was merged (change 3418).
3484. [bug] Some statistics were incorrectly rendered in XML.
[RT #32587]
This includes the following changes:
3326. [func] Added task list statistics: task model, worker
threads, quantum, tasks running, tasks ready.
[RT #27678]
3325. [func] Report cache statistics: memory use, number of
nodes, number of hash buckets, hit and miss counts.
[RT #27056]
3324. [test] Add better tests for ADB stats [RT #27057]
3323. [func] Report the number of buckets the resolver is using.
[RT #27020]
3322. [func] Monitor the number of active TCP and UDP dispatches.
[RT #27055]
3321. [func] Monitor the number of recursive fetches and the
number of open sockets, and report these values in
the statistics channel. [RT #27054]
3320. [func] Added support for monitoring of recursing client
count. [RT #27009]
3319. [func] Added support for monitoring of ADB entry count and
hash size. [RT #27057]
iteration self tuning. The covers nodes clean from
the cache per iteration, nodes written to disk when
rewriting a master file and nodes destroyed per
iteration when destroying a zone or a cache.
[RT #14996]
