* Statistics counter for number of queries dropped by limit on reply addresses
Request list entries can be associated with multiple pending "reply
addresses". Basically each request list entry keeps its own list of
clients that should receive the response once the recursion is finished.
This requires keeping allocations around for each client, and there is
a global limit on the number of *additional* reply addresses that can
be allocated. (Each new request list entry seems to get its own initial
reply address which is not counted against the limit.)
This commit adds a statistics counter "num_queries_replyaddr_limit" that
counts the number of incoming client queries that have been dropped due
to the restriction on allocating additional reply addresses. This allows
distinguishing these drops from other kinds of drops.
* Statistics counter for number of mesh reply entries
Request list entries can be associated with multiple pending "reply
addresses". Since there is a limit on the number of additional reply
addresses that can be allocated which can cause incoming queries to be
dropped if exceeded, it would be nice to be able to track this number.
This commit basically exports the mesh_area's internal counter
`num_reply_addrs` as "threadX.requestlist.current.replies" /
"total.requestlist.current.replies".
* Add extra statistic to track the number of signature validation operations performed by the validator module
* Move validation operation statistic to mesh as suggested
* Fix NULL pointer dereference in case the mesh is not used (and is `NULL`)
Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>
* Fix NULL pointer dereference on qstate and qstate->env in unit test situation
---------
Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>
* v1 EDER poc
* remove superfluous edns_list_get_option function
* create an EDER configurable
* Hackathon 114
* Fixes for version -04
* Generated configparser and configlexer are not versioned in master anymore
* Remove NOERROR DNS Error Reporting; not part of final RFC.
* Use assigned IANA EDNS0 Option Code for Report-Channel.
* Fix buffer protection and agent domain validity
* Use DNS Error Reporting instead of the eder nickname
* Update documentation.
* Fix typo.
* Bail out early if ede is not present.
* Forget previous EDNS options from upstream; this is what was
implicitly happening but not deterministacally.
* Don't report LDNS_EDE_OTHER and bail early if there is no reporting
agent.
* Only do DNS error reporting when a client asked for something that
went wrong.
* Add an error reporting agent in the parent that should be ignored.
* review feedback.
* fixup for fast reload
* Add 'num.dns_error_reports' to stats and test for it.
---------
Co-authored-by: TCY16 <tom@nlnetlabs.nl>
Co-authored-by: Yorgos Thessalonikefs <yorgos@nlnetlabs.nl>
* - fast-reload, add unbound-control fast_reload
* - fast-reload, make a thread to service the unbound-control command.
* - fast-reload, communication sockets for information transfer.
* - fast-reload, fix compile for unbound-dnstap-socket.
* - fast-reload, set nonblocking communication to keep the server thread
responding to DNS requests.
* - fast-reload, poll routine to test for readiness, timeout fails connection.
* - fast-reload, detect loop in sock_poll_timeout routine.
* - fast-reload, send done and exited notification.
* - fast-reload, defines for constants in ipc.
* - fast-reload, ipc socket recv and send resists partial reads and writes and
can continue byte by byte. Also it can continue after an interrupt.
* - fast-reload, send exit command to thread when done.
* - fast-reload, output strings for client on string list.
* - fast-reload, add newline to terminal output.
* - fast-reload, send client string to remote client.
* - fast-reload, better debug output.
* - fast-reload, print queue structure, for output to the remote client.
* - fast-reload, move print items to print queue from fast_reload_thread struct.
* - fast-reload, keep list of pending print queue items in daemon struct.
* - fast-reload, comment explains in_list for printq to print remainder.
* - fast-reload, unit test testdata/fast_reload_thread.tdir that tests the
thread output.
* - fast-reload, fix test link for fast_reload_printq_list_delete function.
* - fast-reload, reread config file from disk.
* - fast-reload, unshare forwards, making the structure locked, with an rwlock.
* - fast-reload, for nonthreaded, the unbound-control commands forward,
forward_add and forward_delete should be distributed to other processes,
but when threaded, they should not be distributed to other threads because
the structure is not thread specific any more.
* - fast-reload, unshared stub hints, making the structure locked, with an rwlock.
* - fast-reload, helpful comments for hints lookup function return value.
* - fast-reload, fix bug in fast reload printout, the strlist appendlist routine,
and printout time statistics after the reload is done.
* - fast-reload, keep track of reloadtime and deletestime and print them.
* - fast-reload, keep track of constructtime and print it.
* - fast-reload, construct new items.
* - fast-reload, better comment.
* - fast-reload, reload the config and swap trees for forwards and stub hints.
* - fast-reload, in forwards_swap_tree set protection of trees with locks.
* - fast-reload, in hints_swap_tree also swap the node count of the trees.
* - fast-reload, reload ipc to stop and start threads.
* - fast-reload, unused forward declarations removed.
* - fast-reload, unit test that fast reload works with forwards and stubs.
* - fast-reload, fix clang analyzer warnings.
* - fast-reload, small documentation entry in unbound-control -h output.
* - fast-reload, printout memory use by fast reload, in bytes.
* - fast-reload, compile without threads.
* - fast-reload, document fast_reload in man page.
* - fast-reload, print ok when done successfully.
* - fast-reload, option for fast-reload commandline, +v verbosity option,
with timing and memory use output.
* - fast-reload, option for fast-reload commandline, +p does not pause threads.
* - fast-reload, option for fast-reload commandline, +d drops mesh queries.
* - fast-reload, fix to poll every thread with nopause to make certain that
resources are not held by the threads and can be deleted.
* - fast-reload, fix to use atomic store for config variables with nopause.
* - fast-reload, reload views.
* - fast-reload, when tag defines are different, it drops the queries.
* - fast-reload, fix tag define check.
* - fast-reload, document that tag change causes drop of queries.
* - fast-reload, fix space in documentation man page.
* - fast-reload, copy respip client information to query state, put views tree
in module env for lookup.
* - fast-reload, nicer respip view comparison.
* - fast-reload, respip global set is in module env.
* - fast-reload, document that respip_client_info acl info is copied.
* - fast-reload, reload the respip_set.
* - fast-reload, document no pause and pick up of use_response_ip boolean.
* - fast-reload, fix test compile.
* - fast-reload, reload local zones.
* Update locking management for iter_fwd and iter_hints methods. (#1054)
fast reload, move most of the locking management to iter_fwd and
iter_hints methods. The caller still has the ability to handle its
own locking, if desired, for atomic operations on sets of different
structs.
Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>
* - fast-reload, reload access-control.
* - fast-reload, reload access control interface, such as interface-action.
* - fast-reload, reload tcp-connection-limit.
* - fast-reload, improve comments on acl_list and tcl_list swap tree.
* - fast-reload, fixup references to old tcp connection limits in open tcp
connections.
* - fast-reload, fixup to clean tcp connection also for different linked order.
* - fast-reload, if no tcp connection limits existed, no need to remove
references for that.
* - fast-reload, document more options that work and do not work.
* - fast-reload, reload auth_zone and rpz data.
* - fast-reload, fix auth_zones_get_mem.
* - fast-reload, fix compilation of testbound for the new comm_timer_get_mem
reference in remote control.
* - fast-reload, change use_rpz with reload.
* - fast-reload, list changes in auth zones and stop zonemd callbacks for
deleted auth zones.
* - fast-reload, note xtree is not swapped, and why it is not swapped.
* - fast-reload, for added auth zones, pick up zone transfer and zonemd tasks.
* - fast-reload, unlock xfr when done with transfer pick up.
* - fast-reload, unlock z when picking up the xfr for it during transfer task
pick up.
* - fast-reload, pick up task changes for added, deleted and modified auth zones.
* - fast-reload, remove xfr of auth zone deletion without tasks.
* - fast-reload, pick up zone transfer config.
* - fast-reload, the main worker thread picks up the transfer tasks and also
performs setup of the xfer struct.
* - fast-reload, keep writelock on newzone when auth zone changes.
* - fast-reload, change cachedb_enabled setting.
* - fast-reload, pick up edns-strings config.
* - fast-reload, note that settings are not updated.
* - fast-reload, pick up dnstap config.
* - fast-reload, dnstap options that need to be loaded without +p.
* - fast-reload, fix auth zone reload
* - fast-reload, remove debug for auth zone test.
* - fast-reload, fix auth zone reload with zone transfer.
* - fast-reload, fix auth zone reload lock order.
* - fast-reload, remove debug from fast reload test.
* - fast-reload, remove unused function.
* - fast-reload, fix the worker trust anchor probe timer lock acquisition in
the probe answer callback routine for trust anchor probes.
* - fast-reload, reload trust anchors.
* - fast-reload, fix trust anchor reload lock on autr global data and test
for trust anchor reload.
* - fast-reload, adjust cache sizes.
* - fast-reload, reload cache sizes when changed.
* - fast-reload, reload validator env changes.
* - fast-reload, reload mesh changes.
* - fast-reload, check for incompatible changes.
* - fast-reload, improve error text for incompatible change.
* - fast-reload, fix check config option compatibility.
* - fast-reload, improve error text for nopause change.
* - fast-reload, fix spelling of incompatible options.
* - fast-reload, reload target-fetch-policy, outbound-msg-retry, max-sent-count
and max-query-restarts.
* - fast-reload, check nopause config change for target-fetch-policy.
* - fast-reload, reload do-not-query-address, private-address and capt-exempt.
* - fast-reload, check nopause config change for do-not-query-address,
private-address and capt-exempt.
* - fast-reload, check fast reload not possible due to interface and
outgoing-interface changes.
* - fast-reload, reload nat64 settings.
* - fast-reload, reload settings stored in the infra structure.
* - fast-reload, fix modstack lookup and remove outgoing-range check.
* - fast-reload, more explanation for config parse failure.
* - fast-reload, reload worker outside network changes.
* - fast-reload, detect incompatible changes in network settings.
* fast-reload, commit test files.
* - fast-reload, fix warnings for call types in windows compile.
* - fast-reload, fix warnings and comm_point_internal for tcp wouldblock calls.
* - fast-reload, extend lock checks for repeat thread ids.
* - fast-reload, additional test cases, cache change and tag changes.
* - fast-reload, fix documentation for auth_zone_verify_zonemd_with_key.
* - fast-reload, fix copy_cfg type casts and memory leak on config parse failure.
* - fast-reload, fix use of WSAPoll.
* Review comments for the fast reload feature (#1259)
* - fast-reload review, respip set can be null from a view.
* - fast-reload review, typos.
* - fast-reload review, keep clang static analyzer happy.
* - fast-reload review, don't forget to copy tag_actions.
* - fast-reload review, less indentation.
* - fast-reload review, don't leak respip_actions when reloading.
* - fast-reload review, protect NULL pointer dereference in get_mem
functions.
* - fast-reload review, add fast_reload_most_options.tdir to test most
options with high verbosity when fast reloading.
* - fast-reload review, don't skip new line on long error printouts.
* - fast-reload review, typo.
* - fast-reload review, use new_z for consistency.
* - fast-reload review, nit for unlock ordering to make eye comparison
with the lock counterpart easier.
* - fast-reload review, in case of error the sockets are already closed.
* - fast-reload review, identation.
* - fast-reload review, add static keywords.
* - fast-reload review, update unbound-control usage text.
* - fast-reload review, updates to the man page.
* - fast-reload, the fast-reload command is experimental.
* - fast-reload, fix compile of doqclient for fast reload functions.
* Changelog comment for #1042
- Merge #1042: Fast Reload. The unbound-control fast_reload is added.
It reads changed config in a thread, then only briefly pauses the
service threads, that keep running. DNS service is only interrupted
briefly, less than a second.
---------
Co-authored-by: Yorgos Thessalonikefs <yorgos@nlnetlabs.nl>
* - dnsoverquic, configure --with-libngtcp2 option.
* - dnsoverquic, create comm_point for doq and receive cmsg local address.
* - dnsoverquic, less obtrusive debug.
* - dnsoverquic, log and fix local port number. Neater subroutines and ifdefs.
* - dnsoverquic, add testcode/doqclient.
* - dnsoverquic, review fixes on doqclient.
* - dnsoverquic, fix unit test testbound link.
* - dnsoverquic, parse query in doqclient.
* - dnsoverquic, link with libngtcp2_crypto_openssl and code for doqclient.
* - dnsoverquic, random routine for doqclient and fix ngaddr allocation, and
check ub_initstate return.
* - dnsoverquic, fix doqclient free of allocated ngaddr addresses.
* - dnsoverquic, enable debug output with -v for doqclient.
* - dnsoverquic, create and set TLS object and TLS context in doqclient.
* - dnsoverquic, work on quic tls context in doqclient.
* - dnsoverquic, set default dnsoverquic port to the standardized 853 port.
* - dnsoverquic, remove debug comment.
* - dnsoverquic, dns-over-quic quic-port: 853 config option.
* - dnsoverquic, log type of interface created at start of unbound.
* - dnsoverquic, log type of no tls https as https when interface is created.
* - dnsoverquic, setup client quic tls methods.
* - dnsoverquic, event work in doqclient.
* - dnsoverquic, explain in documentation that QUIC uses UDP.
* - dnsoverquic, make doqclient exit.
* - dnsoverquic, doqclient cleanup run routine.
* - dnsoverquic, doqclient code nicer.
* - dnsoverquic, doqclient read and timer.
* - dnsoverquic, doqclient write work.
* - dnsoverquic, review fixes.
* - dnsoverquic, detect openssl quic support at configure time.
* - dnsoverquic, do not allow QUIC on port 53 to stop confusion of DoQ and DNS.
* - dnsoverquic, in doqclient, when idle close is returned, drop the connection
without calling ngtcp2_conn_write_connection_close.
* - dnsoverquic, in doqclient, log callbacks.
* - dnsoverquic, in doqclient add extend_max_local_streams_bidi callback.
* - dnsoverquic, in doqclient add client query lists.
* - dnsoverquic, in doqclient, code cleaner, log text nicer.
* - dnsoverquic, in doqclient, work on write_streams.
* - dnsoverquic, in doqclient, use signed int for stream_id, work on the
ngtcp2_recv_stream_data callback.
* - dnsoverquic, in doqclient, print result and fixes for recv data.
* - dnsoverquic, in doqclient, add the event callbacks to fptr wlist.
* - dnsoverquic, in doqclient, when already expired, use zero timeout timer.
* - dnsoverquic, in doqclient, ignore unused return codes from
ngtcp2_conn_writev_stream.
* - dnsoverquic, add doqclient event functions to the unbound-dnstap-socket
test tool for linking.
* - dnsoverquic, in doqclient, fix multiple operands for the commandline.
neater dns message output.
* - dnsoverquic, in doqclient, store packet when write blocks and try later.
* - dnsoverquic, in doqclient, limit number of packets and number of bytes sent.
* - dnsoverquic, in doqclient, better size estimate for outgoing packet.
* - dnsoverquic, in doqclient, fix that already written next packet is not
counted for data length to send.
* - dnsoverquic, in doqclient, early data transmission and session resumption.
* - dnsoverquic, send version negotiation packet.
* - dnsoverquic, send retry and accept the connection.
* - dnsoverquic, storage structures.
* - dnsoverquic, doq connection setup.
* - dnsoverquic, neater code layout for new conn. Fix verbosity of log print.
* - dnsoverquic, doq conn callback functions.
* - dnsoverquic, doq_fill_rand routine in header file.
* - dnsoverquic, keep track of connection ids.
* - dnsoverquic, get_new_connection_id callback.
* - dnsoverquic, create doq_conid tree.
* - dnsoverquic, settings for server connection.
* - dnsoverquic, tls context.
* - dnsoverquic, sendmsg error handling.
* - dnsoverquic, neat code.
* - dnsoverquic, track doq connection last error.
* - dnsoverquic, neater packet address parameters.
* - dnsoverquic, fix uninitialized bytes in msg control in doq sendmsg, and
fix tree cleanup of conid tree.
* - dnsoverquic, better usage text for doqclient.
* - dnsoverquic, neat code.
* - dnsoverquic, connection receive packet handling.
* - dnsoverquic, debug output.
* - dnsoverquic, debug switched meaning of scid and dcid gives
ERR_TRANSPORT_PARAM.
* - dnsoverquic, remove debug output.
* - dnsoverquic, connection delete routine and error from connection read in
more detail with less clutter.
* - dnsoverquic, write to stream, and receive stream data, log packet.
* - dnsoverquic, alpn set up.
* - dnsoverquic, connection close.
* - dnsoverquic, doq_table and locks.
* - dnsoverquic, fix tests.
* - dnsoverquic, better locking.
* - dnsoverquic, doq_stream.
* - dnsoverquic, remove compile warning.
* - dnsoverquic, doq_stream receive data.
* - dnsoverquic, fixes for locks and keep length bytes allocated.
* - dnsoverquic, lock connection on initial insertion.
* - dnsoverquic, reply information, and reply buffer.
* - dnsoverquic, reply info from cache, local-zone and recursion lookups.
* - dnsoverquic, spelling in comment about buffer storage.
* - dnsoverquic, stream write list and doqclient fixes to exit and printout.
* - dnsoverquic, doqclient -q option for short printout.
* - dnsoverquic, unit test with local data reply.
* - dnsoverquic, write connection and write event is set.
* - dnsoverquic, neater logging for write event connection stream writes.
* - dnsoverquic, log remote connection when the streams are written for it.
* - dnsoverquic, better threaded use, threads can write to doq connections at
the same time.
* - dnsoverquic, unit test for the calculation of connection size with a query.
* - dnsoverquic, use less memory per connection.
* - dnsoverquic, remove unit test output.
* - dnsoverquic, add MSG_DONTWAIT so that there is no mistakenly blocking
socket operations.
* - dnsoverquic, doqclient logs address on connection failures.
* - dnsoverquic, compat code for clock get time routine.
* - dnsoverquic, use skip_test for doq unit test.
* - dnsoverquic, fixes for proxyprotocol, use remote_addr and set proxyprotocol
disabled on the doq connection.
* - dnsoverquic, doqclient sets log identity to its name, instead of "unbound".
* - dnsoverquic, handle blocked udp packet writes.
* - dnsoverquic, fix function documentation for verbose_print_addr from
services/listen_dnsport.c.
* - dnsoverquic, fix doq_conn lock protection. The checklock allows to set
the output file name, and doqclient uses that. Print place of lock_protect.
* - dnsoverquic, neater buffer clear when write of blocked packet fails, make
sure that memory area does not overlap for blocked packet addresses when
write of blocked packet fails, and size blocked packet buffer to the pkt buf.
* - dnsoverquic, move lock check after the test to test script in doq test.
* - dnsoverquic, the doq test uses valgrind when enabled.
* - dnsoverquic, git ignore the doqclient test.
* - dnsoverquic, limit the buffer for packets to max packet size with some more.
* - dnsoverquic, spelling fix.
* - dnsoverquic, timer work, structure and adds and deletes.
* - dnsoverquic, timer_tree uses table.lock.
* - dnsoverquic, fix timer tree remove and spelling in header file comment.
* - dnsoverquic, fix testbound for timer compare function linkage.
* - dnsoverquic, timer set add debug output.
* - dnsoverquic, doq_conn_check_timer function.
* - dnsoverquic, doq_done_setup_timer_and_write function.
* - dnsoverquic, fix that doq conn is not deleted whilst editing write and timer.
* - dnsoverquic, Fix#861 make ERROR netevent.h:1073:32: error: field 'blocked_pkt_pi' has incomplete type
* - dnsoverquic, timer element has timeout setup when socket callback complete.
* - dnsoverquic, fix unit test compile.
* - dnsoverquic, timer callback routine, handle timeout and close and delete the
connection if necessary.
* - dnsoverquic, timer pickup stops at current time.
* - dnsoverquic, timer comparable with the event base time.
* - dnsoverquic, erase marked time when timer disabled.
* - dnsoverquic, fix timer to set correctly and lock popped write connection
early, before it is modified.
* - dnsoverquic, fix to unlock connection lock when it is unlinked and deleted.
* - dnsoverquic, fix to unlock connection lock when it is deleted because it is
a duplicate connection.
* - dnsoverquic, fix that doq timer is not disabled when not set.
* - dnsoverquic, quic-size: 8m maximum number of bytes for QUIC buffers.
* - dnsoverquic, flex and bison.
* - dnsoverquic, quic-size turn away new connections when full.
* - dnsoverquic, doqclient outputs stream reset information.
* - dnsoverquic, detect stream close and reset.
* - dnsoverquic, free stream buffers when data is acked and stream is closed.
* - dnsoverquic, delete stream when closed. Unlink it. Allow stream_id 4 as first.
* - dnsoverquic, stats output for mem.quic and num.query.quic.
* - dnsoverquic, review fix.
* - dnsoverquic, fix when compiled without ngtcp2.
* - dnsoverquic, fix to detect ngtcp2_crypto_quictls for openssl crypto, after
change in libngtcp2.
* - dnsoverquic, fix for newer ngtcp2 versions. detect ngtcp2_ccerr_default,
ngtcp2/ngtcp2_crypto_quictls.h, struct ngtcp2_pkt_hd.tokenlen,
struct ngtcp2_settings.tokenlen and struct ngtcp2_version_cid.
* - dnsoverquic, fix for newer ngtcp2 version, detect number of arguments for
ngtcp2_conn_shutdown_stream.
* - dnsoverquic, fix for newer ngtcp2.
* - dnsoverquic, use the functions from util/timeval_func.h.
* - dnsoverquic, fix in doqclient only write transport parameters once.
* - dnsoverquic, debug log output removed.
* - dnsoverquic, fix in doqclient to work with renamed NGTCP2_CC_ALGO_BBR_V2
from ngtcp2.
* - dnsoverquic, fix to check in doq_server_socket_create that tls-service-key
and tls-service-pem have a value.
* - dnsoverquic, fix to error when doq_server_socket_create fails.
* - dnsoverquic, improve linebreaks in configparser additions.
* - dnsoverquic, fix port from interface pickup after main branch change.
* Fix getting user data from SSL, fix calloc warning.
* Fix fwrite return value check in doqclient
* - timeval_substruct from timeval_func.h
- lock_protect also for HAVE_NGTCP2_CCERR_DEFAULT
- fix doq logging for inet_ntop failures
* - memset for consistency
- no value returned from msghdr_get_ecn when S_SPLINT_S is defined
* - dnsoverquic, rerun autoconf.
---------
Co-authored-by: Yorgos Thessalonikefs <yorgos@nlnetlabs.nl>
subnet module deletes global cache and cachedb cache when it
stores a result, and serve-expired is enabled, so that the global
reply, that is older than the ecs reply, does not return after
the ecs reply expires.
Add counter `num_queries_timed_out` meaning queries that were sitting in the
socket queue and waiting to being processed too long. There is no reason
to process such queries, so let's drop it in the very beginning of the
pipeline.
Signed-off-by: Vadim Fedorenko <vadfed@meta.com>
Remove config parser/lexer code as it's rebuilded every time but can
break adding new config options.
Also clean up the code base to avoid mixing actual code changes and lint
issues.
Signed-off-by: Vadim Fedorenko <vadfed@meta.com>
Create server cookies for clients that send client cookies.
Needs to be turned on in the config file with:
answer-cookie: yes
A cookie-secret can be configured for anycast setups.
Also adds an access control list that will allow queries with
either a valid cookie or over a stateful transport.
