Under the overmem conditions, the header could get unlinked from the
SIEVE LRU using a different path. This could lead to double-unlink
which causes assertion failure. Add a guard to ISC_SIEVE_UNLINK() to
unlink only still linked headers.
Changes introduced by 72862c2abc moved the
default configuration from within `bin/named` to a central place
`bin/includes`.
The default configuration is conditioned by several compile-time macro.
While for most of them it's fine because they are defined in the global
`config.h` file included by default to all binaries (by meson), one
specific is not defined here. `HAVE_SO_REUSEPORT_LB` was defined in
`lib/isc/include/isc/netmgr.h` which is of course not included in
`bin/includes/defaultconfig.h`.
As a result, reuseport was disabled for all platform by default, even
the supported ones. This fixes the problem by checking if reuseport is
available on the platform from meson `config.h` generation directly,
which makes `HAVE_SO_REUSEPORT_LB` available everywhere.
The sun_path field is not used anymore, and consumes over a hundred
bytes for every isc_netaddr_t object. Remove it.
As isc_netaddr_t is used in cfg_obj_t, in some huge configuration trees
(e.g., a million zones), the gain is almost 1GB of resident memory.
When the arc4random_uniform() is called on NetBSD with upper_bound that
makes no sense statistically (0 or 1), the call crashes the calling
program. Fix this by returning 0 when upper bound is < 2 as does Linux,
FreeBSD and NetBSD. (Hint: System CSPRNG should never crash.)
Add a magic number check to ensure the memory context validity before
destorying it.
This check is needed now as it was done before implicitly when
isc_mem_inuse was called, but isc_mem_inuse is now called later (to be
able to dump the outstanding allocations).
When a memory context is destroyed, if the `checkfree` property is set,
the program assert there is no remaining allocation. If there are and
assertions are enabled, the program immediately stops.
However, if memory trace/record debug is enabled, the dump of
outstanding allocation won't be printed as it is done after the
no remaining allocation assertion check.
This moves the no remaining allocation assertion check after the dump of
outstanding allocations, so it is still possible to figure out what's
still allocated by this memory context.
if a zone reload is already in progress when 'rndc reload <zone>' is
run, currently the message returned in "zone reload queued", which
is correct, but it's identical to the message returned when a reload
was *not* in progress, so the user can't easily tell what happened.
a user could reload a zone twice and not realize that only one
reload actually took place.
this has been addressed by changing the message returned to
"zone reload was already queued".
a new result code ISC_R_LOADING has been added to signal this
condition, taking the place of ISC_R_RELOAD, which was obsolete
and has been removed.
Use arc4random on platforms where available. arc4random() provides high
quality cryptographically-secure pseudo-random numbers and is generally
recommended for application use.
The uv_random() call unfortunately uses getentropy() on platforms like
MacOS, OpenBSD or NetBSD which is not recommended for application use.
It was discovered in an upcoming academic paper that a xoshiro128**
internal state can be recovered by an external 3rd party allowing to
predict UDP ports and DNS IDs in the outgoing queries. This could lead
to an attacker spoofing the DNS answers with great efficiency and
poisoning the DNS cache.
Change the internal random generator to system CSPRNG with buffering to
avoid excessive syscalls.
Thanks Omer Ben Simhon and Amit Klein of Hebrew University of Jerusalem
for responsibly reporting this to us. Very cool research!
Functions hex_decode_init(), hex_decode_char() and hex_decode_finish()
are now exposed, as well as the context hex_decode_ctx_t. They now are
respectively called isc_hex_decodeinit(), isc_hex_decodechar(),
isc_hex_decodefinish() and isc_hex_decodectx_t.
This enable to re-implement the functionality of isc_hex_decodestring()
in contextes where the input is not a NULL-terminated string, but, for
example, individual characters extracted (and avoid creating an
intermediate buffer to store them). This also enable to decode a stream
of hex characters where only hex characters are expected (i.e. no white
spaces).
Enabling LTO in the subsequent commit requires the file names to be
unique and having same probes.d in each of the libraries breaks this
requirement. Rename probes.d to probes-{isc,dns,ns}.d files and adjust
the includes.
Previously, the fetch contexts were stored inside rwlocked hashmap
table. This was one of the most contended places for the resolver,
especially in the cold cache situation.
Replace the locked hashmap with the lock-free hashtable from the RCU
library and protect the fetch contexts against reuse by replacing the
libisc reference counting with urcu_ref that can soft-fail in situation
where the reference count is already zero. This allows us to easily
skip re-using the fetch context if it is already in process of being
destroyed.
This is the first commit in series that aims to reduce the node locking
by replacing the single-linked list of slabtop(s) with CDS linked list.
This commit doesn't do anything else beyond replacing .next link with
the cds_list_head. RCU semantics is going to be added in the subsequent
commits.
Clang 20 is complaining about passing NULL to an argument with 'nonnull'
attribute. Mark these two functions with the same attribute to assure
that these two function also don't accept NULL as an argument.
libuv expects file descriptors <= STDERR_FILENO are in use. otherwise,
it may abort when closing a file descriptor it opened.
See https://github.com/libuv/libuv/pull/4559Closes#5226
In gcc 15, __builtin_stdc_rotate_{left,right} was added. Use these
builtins when available otherwise rewrite the ISC_ROTATE_LEFT and
ISC_ROTATE_RIGHT using _Generic.
Use C23 stdckdint.h when available and define ckd_{mul,add,sub} shims to
__builtin_{mul,add,sub}_overflow(). Require the __builtin functions
unconditionally.
Currently following __builtin functions are used:
__builtin_add_overflow
__builtin_mul_overflow
__builtin_prefetch
__builtin_sub_overflow
__builtin_unreachable
These are generally available on our supported platform, and also we use
some of these unconditionally anyway in qp.c. Thus make the support for
these functions mandatory so we fail early in the 'setup' step.
The fxhash implementation was missing a constant for 32-bit platforms.
This has been fixed. Constant for 64-bit platform was update to match
the current Rust constants.
The following check:
__builtin_types_compatible_p(size_t, uint64_t)
doesn't work with default compiler on macOS. Workaround the issue
by typing the size_t to matching unsigned int type.
Using `static inline` functions in the headers break gcov as it cannot
properly track the hits. To fix the issue, convert the expressions to
statement macros. The added static assertions will ensure integer
promotion cannot occur unlike its previous function counterpart.
The `<isc/bit.h>` header is a GNU C11 compatible version of C23's
`<stdbit.h>`.
It currently uses either `<stdbit.h>` or the equivilent compiler
builtins. However, the generic `__builtin_ctzg` and `__builtin_ctlz`
builtins are not available in every compiler version and thus falls
back to manually selecting from type.
Furthermore, the ctz fallback has been removed since `__builtin_ctzll`
has been used for a while directly without any compilation issues from
users. Thus, we can also require `__builtin_ctz`.
Unlike the rest of C23's bit utilities, we avoid the stdc_rotate_*
functions since we don't need the rotation modulus precision. This adds
a couple (admittedly cheap) unwanted instructions on some architectures.
> Put a space before opening parentheses only after control statement
> keywords (for/if/while...) except this option doesn’t apply to ForEach
> and If macros. This is useful in projects where ForEach/If macros are
> treated as function calls instead of control statements.
> Insert braces after control statements (if, else, for, do, and while)
> in C++ unless the control statements are inside macro definitions or
> the braces would enclose preprocessor directives.
The default stack sizes varies between operating systems and between
different system libc libraries from 128kB (Alpine Linux with MUSL) to
8M (Linux with glibc). Document the different values used to justify
the value of THREAD_MINSTACKSIZE (currently set to 1MB).
When running the isc_quota unit test with less than usual amount of
RAM (e.g. in a CI for architectures with 32 bits of address space),
the pthread_create() function fails with the "Resource temporarily
unavailable (11):" error code.
Add functions to get and set the thread stack size (if requested),
and use these to set the thread stack size to smaller value in the
isc_quota unit test.
There is a data race when QP is reclaiming chunks on the call_rcu
threads and it tries to log the number of reclaimed chunks while the
server is shuttingdown. Workaround this by adding rcu_barrier() before
shuttingdown the global log context.
This required couple of internal changes to the isc_mem_debugging.
The isc_mem_debugging is now internal to isc_mem unit and there are
three new functions:
1. isc_mem_setdebugging() can change the debugging setting for an
individual memory context. This is need for the memory contexts used
for OpenSSL, libxml and libuv accounting as recording and tracing
memory is broken there.
2. isc_mem_debugon() / isc_mem_debugoff() can be used to change default
memory debugging flags as well as debugging flags for isc_g_mctx.
Additionally, the memory debugging is inconsistent across the code-base.
For now, we are keeping the existing flags, but three new environment
variables have been added 'ISC_MEM_DEBUGRECORD', 'ISC_MEM_DEBUGTRACE'
and 'ISC_MEM_DEBUGUSAGE' to set the global debugging flags at any
program using the memory contexts.
Instead of having individual memory contexts scattered across different
files and called different names, add a single memory context called
isc_g_mctx that replaces named_g_mctx and various other global memory
contexts in various utilities and tests.
There is only a single network manager running on top of the loop
manager (except for tests). Refactor the network manager to be a
singleton (a single instance) and change the unit tests, so that the
shorter read timeouts apply only to a specific handle, not the whole
extra 'connect_nm' network manager instance.
All the applications built on top of the loop manager were required to
create just a single instance of the loop manager. Refactor the loop
manager to not expose this instance to the callers and keep the loop
manager object internal to the isc_loop compilation unit.
This significantly simplifies a number of data structures and calls to
the isc_loop API.
Instead of having hand crafted attach/detach/destroy functions, replace
them with the standard ISC_REFCOUNT macro. This also have advantage
that delayed netmgr detach (from dns_dispatch) now doesn't cause
assertion failure. This can happen with delayed (call_rcu) shutdown of
dns_adb.
The dns_adb cleaning is little bit muddled as it mixes the "TTL"
based cleaning (.expire_v4 and .expire_v6 for adbname, .expires for
adbentry) with overmem cleaning.
Rewrite the LRU based cleaning to use SIEVE algorithm and to be overmem
cleaning only with a requirement to always cleanup at least 2-times the
size of the newly added entry.
The .inuse member was causing a lot of contention between threads using
the same memory context. Scather the .inuse and .overmem members of
isc_mem_t structure to be an per-tid array of variables to reduce the
contention as the writes are now independent of each other.
The array uses one tad bit nasty trick, as ISC_TID_UNKNOWN is now -1,
the array has been sized to fit the unknown tid with [-1] index into the
array accomplished with `ctx->stat = &ctx->stat_s[1];`. It will not win
a beauty contest, but it works seamlessly by just passing `isc_tid()` as
an index into the array.
The caveat here is that gathering the real inuse value requires walking
the whole array for all registered tid values (isc_tid_count()). The
gather part happens only when statistics are being gathered or when
isc_mem_isovermem() is called. As the isc_mem_isovermem() call happens
only when new data is being added to cache or ADB, it doesn't happen on
the hottest (read-only) path and according to the measurements, it
doesn't slow down neither the cold cache nor the hot cache latency.
As POSIX guarantees only that the type ssize_t shall be capable of
storing values at least in the range [-1, {SSIZE_MAX}], it can't be used
to calculate the difference between two memory sizes. Change the logic
for junk filling to test whether the new size is larger than old size
and then use size_t as the result will be always positive.
The jemalloc arena in isc_mem was added to solve runaway memory problem
for outgoing TCP connections. In the end, this was a red herring and
the jemalloc arena code is now unused (via e28266bf). Remove the
support for jemalloc memory arenas as we can restore this at any time if
we need it ever again, but right now it's just a dead code.
In jemalloc_shim.h, we relied on including <isc/overflow.h> implicitly
instead of explicitly and same was happening inside isc/overflow.h - the
stdbool.h (for bool type) was being included implicitly instead of
explicitly.
Change the internal type used for isc_tid unit to isc_tid_t to hide the
specific integer type being used for the 'tid'. Internally, the signed
integer type is being used. This allows us to have negatively indexed
arrays that works both for threads with assigned tid and the threads
with unassigned tid. This should be used only in specific situations.
