Clang 20 is complaining about passing NULL to an argument with 'nonnull'
attribute. Mark these two functions with the same attribute to assure
that these two function also don't accept NULL as an argument.
In gcc 15, __builtin_stdc_rotate_{left,right} was added. Use these
builtins when available otherwise rewrite the ISC_ROTATE_LEFT and
ISC_ROTATE_RIGHT using _Generic.
Use C23 stdckdint.h when available and define ckd_{mul,add,sub} shims to
__builtin_{mul,add,sub}_overflow(). Require the __builtin functions
unconditionally.
Currently following __builtin functions are used:
__builtin_add_overflow
__builtin_mul_overflow
__builtin_prefetch
__builtin_sub_overflow
__builtin_unreachable
These are generally available on our supported platform, and also we use
some of these unconditionally anyway in qp.c. Thus make the support for
these functions mandatory so we fail early in the 'setup' step.
The fxhash implementation was missing a constant for 32-bit platforms.
This has been fixed. Constant for 64-bit platform was update to match
the current Rust constants.
The following check:
__builtin_types_compatible_p(size_t, uint64_t)
doesn't work with default compiler on macOS. Workaround the issue
by typing the size_t to matching unsigned int type.
Using `static inline` functions in the headers break gcov as it cannot
properly track the hits. To fix the issue, convert the expressions to
statement macros. The added static assertions will ensure integer
promotion cannot occur unlike its previous function counterpart.
The `<isc/bit.h>` header is a GNU C11 compatible version of C23's
`<stdbit.h>`.
It currently uses either `<stdbit.h>` or the equivilent compiler
builtins. However, the generic `__builtin_ctzg` and `__builtin_ctlz`
builtins are not available in every compiler version and thus falls
back to manually selecting from type.
Furthermore, the ctz fallback has been removed since `__builtin_ctzll`
has been used for a while directly without any compilation issues from
users. Thus, we can also require `__builtin_ctz`.
Unlike the rest of C23's bit utilities, we avoid the stdc_rotate_*
functions since we don't need the rotation modulus precision. This adds
a couple (admittedly cheap) unwanted instructions on some architectures.
When running the isc_quota unit test with less than usual amount of
RAM (e.g. in a CI for architectures with 32 bits of address space),
the pthread_create() function fails with the "Resource temporarily
unavailable (11):" error code.
Add functions to get and set the thread stack size (if requested),
and use these to set the thread stack size to smaller value in the
isc_quota unit test.
This required couple of internal changes to the isc_mem_debugging.
The isc_mem_debugging is now internal to isc_mem unit and there are
three new functions:
1. isc_mem_setdebugging() can change the debugging setting for an
individual memory context. This is need for the memory contexts used
for OpenSSL, libxml and libuv accounting as recording and tracing
memory is broken there.
2. isc_mem_debugon() / isc_mem_debugoff() can be used to change default
memory debugging flags as well as debugging flags for isc_g_mctx.
Additionally, the memory debugging is inconsistent across the code-base.
For now, we are keeping the existing flags, but three new environment
variables have been added 'ISC_MEM_DEBUGRECORD', 'ISC_MEM_DEBUGTRACE'
and 'ISC_MEM_DEBUGUSAGE' to set the global debugging flags at any
program using the memory contexts.
Instead of having individual memory contexts scattered across different
files and called different names, add a single memory context called
isc_g_mctx that replaces named_g_mctx and various other global memory
contexts in various utilities and tests.
There is only a single network manager running on top of the loop
manager (except for tests). Refactor the network manager to be a
singleton (a single instance) and change the unit tests, so that the
shorter read timeouts apply only to a specific handle, not the whole
extra 'connect_nm' network manager instance.
All the applications built on top of the loop manager were required to
create just a single instance of the loop manager. Refactor the loop
manager to not expose this instance to the callers and keep the loop
manager object internal to the isc_loop compilation unit.
This significantly simplifies a number of data structures and calls to
the isc_loop API.
Instead of having hand crafted attach/detach/destroy functions, replace
them with the standard ISC_REFCOUNT macro. This also have advantage
that delayed netmgr detach (from dns_dispatch) now doesn't cause
assertion failure. This can happen with delayed (call_rcu) shutdown of
dns_adb.
The dns_adb cleaning is little bit muddled as it mixes the "TTL"
based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for
adbentry) with overmem cleaning.
Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem
cleaning only with a requirement to always cleanup at least 2-times the
size of the newly added entry.
The jemalloc arena in isc_mem was added to solve runaway memory problem
for outgoing TCP connections. In the end, this was a red herring and
the jemalloc arena code is now unused (via e28266bf). Remove the
support for jemalloc memory arenas as we can restore this at any time if
we need it ever again, but right now it's just a dead code.
In jemalloc_shim.h, we relied on including <isc/overflow.h> implicitly
instead of explicitly and same was happening inside isc/overflow.h - the
stdbool.h (for bool type) was being included implicitly instead of
explicitly.
Change the internal type used for isc_tid unit to isc_tid_t to hide the
specific integer type being used for the 'tid'. Internally, the signed
integer type is being used. This allows us to have negatively indexed
arrays that works both for threads with assigned tid and the threads
with unassigned tid. This should be used only in specific situations.
commandline.c failed to compile on Solaris because NAME_MAX was
undefined. Include 'isc/dir.h' which defines NAME_MAX for platforms
that don't define it.
In file included from commandline.c:54:
./include/isc/commandline.h:31:38: error: 'NAME_MAX' undeclared here (not in a function)
31 | extern char isc_commandline_progname[NAME_MAX];
| ^~~~~~~~
Using C23 attributes for `counted_by` is broken with clang.
`__has_attribute` is used since `__has_c_attribute` only works with C23
attributes, (`gnu::counted_by`/`clang::counted_by`)
There were several methods how we used 'argv[0]'. Some programs had a
static value, some programs did use isc_file_progname(), some programs
stripped 'lt-' from the beginning of the name. And some used argv[0]
directly.
Unify the handling and all the variables into isc_commandline_progname
that gets populated by the new isc_commandline_init(argc, argv) call.
Instead of giving the memory pools names with an explicit call to
isc_mempool_setname(), add the name to isc_mempool_create() call to have
all the memory pools an unconditional name.
Instead of giving the memory context names with an explicit call to
isc_mem_setname(), add the name to isc_mem_create() call to have all the
memory contexts an unconditional name.
previously, ISC_LIST_FOREACH and ISC_LIST_FOREACH_SAFE were
two separate macros, with the _SAFE version allowing entries
to be unlinked during the loop. ISC_LIST_FOREACH is now also
safe, and the separate _SAFE macro has been removed.
similarly, the ISC_LIST_FOREACH_REV macro is now safe, and
ISC_LIST_FOREACH_REV_SAFE has also been removed.
qp-tries allocate their nodes (twigs) in chunks to reduce allocator
pressure and improve memory locality. The choice of chunk size presents
a tradeoff: larger chunks benefit qp-tries with many values (as seen
in large zones and resolvers) but waste memory in smaller use cases.
Previously, our fixed chunk size of 2^10 twigs meant that even an
empty qp-trie would consume 12KB of memory, while reducing this size
would negatively impact resolver performance.
This commit implements an adaptive chunking strategy that:
- Tracks the size of the most recently allocated chunk.
- Doubles the chunk size for each new allocation until reaching a
predefined maximum.
This approach effectively balances memory efficiency for small tries
while maintaining the performance benefits of larger chunk sizes for
bigger data structures.
This commit also splits the callback freeing qpmultis into two
phases, one that frees the underlying qptree, and one that reclaims
the qpmulti memory. In order to prevent races between the qpmulti
destructor and chunk garbage collection jobs, the second phase is
protected by reference counting.
Previously all kinds of TCP timeouts had a single getter and setter
functions. Separate each timeout to its own getter/setter functions,
because in majority of cases only one is required at a time, and it's
not optimal expanding those functions every time a new timeout value
is implemented.
The new 'tcp-primaries-timeout' configuration option works the same way
as the existing 'tcp-initial-timeout' option, but applies only to the
TCP connections made to the primary servers, so that the timeout value
can be set separately for them. The default is 15 seconds.
Also, while accommodating zone.c's code to support the new option, make
a light refactoring with the way UDP timeouts are calculated by using
definitions instead of hardcoded values.
ISC_LIST_FOREACH and related macros now use 'typeof(list.head)' to
declare the list elements automatically; the caller no longer needs
to do so.
ISC_LIST_FOREACH_SAFE also now implicitly declares its own 'next'
pointer, so it only needs three parameters instead of four.
Add a function that checks if a 'hostname' is not a valid IPv4 or IPv6
address. Returns 'true' if the hostname is likely a domain name, and
'false' if it represents an IP address.
This is the core implementation of the SIEVE algorithm described in the
following paper:
Zhang, Yazhuo, Juncheng Yang, Yao Yue, Ymir Vigfusson, and K V
Rashmi. “SIEVE Is Simpler than LRU: An Efficient Turn-Key Eviction
Algorithm for Web Caches,” n.d.. available online from
https://junchengyang.com/publication/nsdi24-SIEVE.pdf
This can be set at the option, view and server levels and causes
named to add an EDNS ZONEVERSION option to requests. Replies are
logged to the 'zoneversion' category.
This merge request resolves some performance regressions introduced
with the change from isc_symtab_t to isc_hashmap_t.
The key improvements are:
1. Using a faster hash function than both isc_hashmap_t and
isc_symtab_t. The previous implementation used SipHash, but the
hashflood resistance properties of SipHash are unneeded for config
parsing.
2. Shrinking the initial size of the isc_hashmap_t used inside
isc_symtab_t. Symtab is mainly used for config parsing, and the
when used that way it will have between 1 and ~50 keys, but the
previous implementation initialized a map with 128 slots.
By initializing a smaller map, we speed up mallocs and optimize for
the typical case of few config keys.
3. Slight optimization of the string matching in the hashmap, so that
the tail is handled in a single load + comparison, instead of byte
by byte.
Of the three improvements, this is the least important.
The isc_mem API is one of the most commonly used APIs that didn't
used ISC_REFCOUNT_DECL and ISC_REFCOUNT_IMPL macros. Replace the
implementation of isc_mem_attach(), isc_mem_detach() and
isc_mem_destroy() with the respective macros.
This also removes the legacy isc_mem_destroy() functionality that would
check whether all references had been detached from the memory context
as it doesn't work reliably when using the call_rcu() API. Instead of
doing this individually, call isc_mem_checkdestroyed(stderr) from the
isc_mem_destroy() macro to keep the extra check that all contexts were
freed when the program is exiting.
Previously, a gcc < 4.6 shim for _Static_assert() was included. Such an
old compiler is not supported now anyway, so the macro variant has been
removed in favor of a single definition using _Static_assert().
Previously, the LOCK()/UNLOCK() and friends macros were defined in the
isc/util.h header. Those macros were moved to their respective headers
as those would have to be included anyway if that particular lock was in
use.
Formerly, isc/util.h would pull a few extra headers (isc/list.h,
isc/attributes.h, isc/result.h and errno.h). These includes were
removed in favor of including them directly when used.
The short convenience list macros were used very sparingly and
inconsistenly in the code base. As the consistency is prefered over
the convenience, all shortened list macro were removed in favor of
their ISC_LIST API targets.
Since algorithm fetching is handled purely in libisc, FIPS mode toggling
can be purely done in within the library instead of provider fetching in
the binary for OpenSSL >=3.0.
Disabling FIPS mode isn't a realistic requirement and isn't done
anywhere in the codebase. Make the FIPS mode toggle enable-only to
reflect the situation.
Instead of relying on unreliable order of execution of the library
constructors and destructors, move them to individual binaries. The
advantage is that the execution time and order will remain constant and
will not depend on the dynamic load dependency solver.
This requires more work, but that was mitigated by a simple requirement,
any executable using libisc and libdns, must include <isc/lib.h> and
<dns/lib.h> respectively (in this particular order). In turn, these two
headers must not be included from within any library as they contain
inlined functions marked with constructor/destructor attributes.
In the next commit, we need to know whether the timer has been started
or stopped. Add isc_timer_running() function that returns true if the
timer has been started.
The isc_counter_create() doesn't need the return value (it was always
ISC_R_SUCCESS), use the macros to implement the reference counting,
little style cleanup, and expand the unit test.
Running jobs which were entered into the isc_quota queue is the
responsibility of the isc_quota_release() function, which, when
releasing a previously acquired quota, checks whether the queue
is empty, and if it's not, it runs a job from the queue without touching
the 'quota->used' counter. This mechanism is susceptible to a possible
hangup of a newly queued job in case when between the time a decision
has been made to queue it (because used >= max) and the time it was
actually queued, the last quota was released. Since there is no more
quotas to be released (unless arriving in the future), the newly
entered job will be stuck in the queue.
Fix the wrong memory ordering for 'quota->used', as the relaxed
ordering doesn't ensure that data modifications made by one thread
are visible in other threads.
Add checks in both isc_quota_release() and isc_quota_acquire_cb()
to make sure that the described hangup does not happen. Also see
code comments.
