Commit graph

32 commits

Author SHA1 Message Date
Wouter Wijngaards
218f5cfc92
Fast Reload Option (#1042)
* - fast-reload, add unbound-control fast_reload

* - fast-reload, make a thread to service the unbound-control command.

* - fast-reload, communication sockets for information transfer.

* - fast-reload, fix compile for unbound-dnstap-socket.

* - fast-reload, set nonblocking communication to keep the server thread
  responding to DNS requests.

* - fast-reload, poll routine to test for readiness, timeout fails connection.

* - fast-reload, detect loop in sock_poll_timeout routine.

* - fast-reload, send done and exited notification.

* - fast-reload, defines for constants in ipc.

* - fast-reload, ipc socket recv and send resists partial reads and writes and
  can continue byte by byte. Also it can continue after an interrupt.

* - fast-reload, send exit command to thread when done.

* - fast-reload, output strings for client on string list.

* - fast-reload, add newline to terminal output.

* - fast-reload, send client string to remote client.

* - fast-reload, better debug output.

* - fast-reload, print queue structure, for output to the remote client.

* - fast-reload, move print items to print queue from fast_reload_thread struct.

* - fast-reload, keep list of pending print queue items in daemon struct.

* - fast-reload, comment explains in_list for printq to print remainder.

* - fast-reload, unit test testdata/fast_reload_thread.tdir that tests the
  thread output.

* - fast-reload, fix test link for fast_reload_printq_list_delete function.

* - fast-reload, reread config file from disk.

* - fast-reload, unshare forwards, making the structure locked, with an rwlock.

* - fast-reload, for nonthreaded, the unbound-control commands forward,
  forward_add and forward_delete should be distributed to other processes,
  but when threaded, they should not be distributed to other threads because
  the structure is not thread specific any more.

* - fast-reload, unshared stub hints, making the structure locked, with an rwlock.

* - fast-reload, helpful comments for hints lookup function return value.

* - fast-reload, fix bug in fast reload printout, the strlist appendlist routine,
  and printout time statistics after the reload is done.

* - fast-reload, keep track of reloadtime and deletestime and print them.

* - fast-reload, keep track of constructtime and print it.

* - fast-reload, construct new items.

* - fast-reload, better comment.

* - fast-reload, reload the config and swap trees for forwards and stub hints.

* - fast-reload, in forwards_swap_tree set protection of trees with locks.

* - fast-reload, in hints_swap_tree also swap the node count of the trees.

* - fast-reload, reload ipc to stop and start threads.

* - fast-reload, unused forward declarations removed.

* - fast-reload, unit test that fast reload works with forwards and stubs.

* - fast-reload, fix clang analyzer warnings.

* - fast-reload, small documentation entry in unbound-control -h output.

* - fast-reload, printout memory use by fast reload, in bytes.

* - fast-reload, compile without threads.

* - fast-reload, document fast_reload in man page.

* - fast-reload, print ok when done successfully.

* - fast-reload, option for fast-reload commandline, +v verbosity option,
  with timing and memory use output.

* - fast-reload, option for fast-reload commandline, +p does not pause threads.

* - fast-reload, option for fast-reload commandline, +d drops mesh queries.

* - fast-reload, fix to poll every thread with nopause to make certain that
  resources are not held by the threads and can be deleted.

* - fast-reload, fix to use atomic store for config variables with nopause.

* - fast-reload, reload views.

* - fast-reload, when tag defines are different, it drops the queries.

* - fast-reload, fix tag define check.

* - fast-reload, document that tag change causes drop of queries.

* - fast-reload, fix space in documentation man page.

* - fast-reload, copy respip client information to query state, put views tree
  in module env for lookup.

* - fast-reload, nicer respip view comparison.

* - fast-reload, respip global set is in module env.

* - fast-reload, document that respip_client_info acl info is copied.

* - fast-reload, reload the respip_set.

* - fast-reload, document no pause and pick up of use_response_ip boolean.

* - fast-reload, fix test compile.

* - fast-reload, reload local zones.

* Update locking management for iter_fwd and iter_hints methods. (#1054)

fast reload, move most of the locking management to iter_fwd and
iter_hints methods. The caller still has the ability to handle its
own locking, if desired, for atomic operations on sets of different
structs.

Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>

* - fast-reload, reload access-control.

* - fast-reload, reload access control interface, such as interface-action.

* - fast-reload, reload tcp-connection-limit.

* - fast-reload, improve comments on acl_list and tcl_list swap tree.

* - fast-reload, fixup references to old tcp connection limits in open tcp
  connections.

* - fast-reload, fixup to clean tcp connection also for different linked order.

* - fast-reload, if no tcp connection limits existed, no need to remove
  references for that.

* - fast-reload, document more options that work and do not work.

* - fast-reload, reload auth_zone and rpz data.

* - fast-reload, fix auth_zones_get_mem.

* - fast-reload, fix compilation of testbound for the new comm_timer_get_mem
  reference in remote control.

* - fast-reload, change use_rpz with reload.

* - fast-reload, list changes in auth zones and stop zonemd callbacks for
  deleted auth zones.

* - fast-reload, note xtree is not swapped, and why it is not swapped.

* - fast-reload, for added auth zones, pick up zone transfer and zonemd tasks.

* - fast-reload, unlock xfr when done with transfer pick up.

* - fast-reload, unlock z when picking up the xfr for it during transfer task
  pick up.

* - fast-reload, pick up task changes for added, deleted and modified auth zones.

* - fast-reload, remove xfr of auth zone deletion without tasks.

* - fast-reload, pick up zone transfer config.

* - fast-reload, the main worker thread picks up the transfer tasks and also
  performs setup of the xfer struct.

* - fast-reload, keep writelock on newzone when auth zone changes.

* - fast-reload, change cachedb_enabled setting.

* - fast-reload, pick up edns-strings config.

* - fast-reload, note that settings are not updated.

* - fast-reload, pick up dnstap config.

* - fast-reload, dnstap options that need to be loaded without +p.

* - fast-reload, fix auth zone reload

* - fast-reload, remove debug for auth zone test.

* - fast-reload, fix auth zone reload with zone transfer.

* - fast-reload, fix auth zone reload lock order.

* - fast-reload, remove debug from fast reload test.

* - fast-reload, remove unused function.

* - fast-reload, fix the worker trust anchor probe timer lock acquisition in
  the probe answer callback routine for trust anchor probes.

* - fast-reload, reload trust anchors.

* - fast-reload, fix trust anchor reload lock on autr global data and test
  for trust anchor reload.

* - fast-reload, adjust cache sizes.

* - fast-reload, reload cache sizes when changed.

* - fast-reload, reload validator env changes.

* - fast-reload, reload mesh changes.

* - fast-reload, check for incompatible changes.

* - fast-reload, improve error text for incompatible change.

* - fast-reload, fix check config option compatibility.

* - fast-reload, improve error text for nopause change.

* - fast-reload, fix spelling of incompatible options.

* - fast-reload, reload target-fetch-policy, outbound-msg-retry, max-sent-count
  and max-query-restarts.

* - fast-reload, check nopause config change for target-fetch-policy.

* - fast-reload, reload do-not-query-address, private-address and capt-exempt.

* - fast-reload, check nopause config change for do-not-query-address,
  private-address and capt-exempt.

* - fast-reload, check fast reload not possible due to interface and
  outgoing-interface changes.

* - fast-reload, reload nat64 settings.

* - fast-reload, reload settings stored in the infra structure.

* - fast-reload, fix modstack lookup and remove outgoing-range check.

* - fast-reload, more explanation for config parse failure.

* - fast-reload, reload worker outside network changes.

* - fast-reload, detect incompatible changes in network settings.

* fast-reload, commit test files.

* - fast-reload, fix warnings for call types in windows compile.

* - fast-reload, fix warnings and comm_point_internal for tcp wouldblock calls.

* - fast-reload, extend lock checks for repeat thread ids.

* - fast-reload, additional test cases, cache change and tag changes.

* - fast-reload, fix documentation for auth_zone_verify_zonemd_with_key.

* - fast-reload, fix copy_cfg type casts and memory leak on config parse failure.

* - fast-reload, fix use of WSAPoll.

* Review comments for the fast reload feature (#1259)

* - fast-reload review, respip set can be null from a view.

* - fast-reload review, typos.

* - fast-reload review, keep clang static analyzer happy.

* - fast-reload review, don't forget to copy tag_actions.

* - fast-reload review, less indentation.

* - fast-reload review, don't leak respip_actions when reloading.

* - fast-reload review, protect NULL pointer dereference in get_mem
  functions.

* - fast-reload review, add fast_reload_most_options.tdir to test most
  options with high verbosity when fast reloading.

* - fast-reload review, don't skip new line on long error printouts.

* - fast-reload review, typo.

* - fast-reload review, use new_z for consistency.

* - fast-reload review, nit for unlock ordering to make eye comparison
  with the lock counterpart easier.

* - fast-reload review, in case of error the sockets are already closed.

* - fast-reload review, identation.

* - fast-reload review, add static keywords.

* - fast-reload review, update unbound-control usage text.

* - fast-reload review, updates to the man page.

* - fast-reload, the fast-reload command is experimental.

* - fast-reload, fix compile of doqclient for fast reload functions.

* Changelog comment for #1042
- Merge #1042: Fast Reload. The unbound-control fast_reload is added.
  It reads changed config in a thread, then only briefly pauses the
  service threads, that keep running. DNS service is only interrupted
  briefly, less than a second.

---------

Co-authored-by: Yorgos Thessalonikefs <yorgos@nlnetlabs.nl>
2025-03-31 15:25:24 +02:00
W.C.A. Wijngaards
c3dd6a2dbd - Add dnstap-sample-rate that logs only 1/N messages, for high volume
server environments. Thanks Dan Luther.
2024-07-19 10:04:40 +02:00
k-akashi
4b9cd8e81d Add DoH and DoT to dnstap message 2023-12-27 07:26:21 +09:00
W.C.A. Wijngaards
d8bd9845cc - Fix dnstap that assertion failed on logging other than UDP and TCP
traffic. It lists it as TCP traffic.
2023-12-05 13:14:08 +01:00
W.C.A. Wijngaards
2a2598dbf2 - Fix #888: [FR] Use kernel timestamps for dnstap. 2023-05-16 08:50:38 +02:00
George Thessalonikefs
64546fcfb4 - Fix unused variable warning when compiling with --enable-dnstap. 2021-06-08 22:15:17 +02:00
W.C.A. Wijngaards
af96ff157e Fix leak of socket mem on failure and fix if tcp has to wait we
do not know the interface at the time.  Later when it knows the
interface we do not have 'sq' with the zone name.
2020-12-09 13:52:49 +01:00
W.C.A. Wijngaards
31cedb47cb Remove unused whitespace, add missing header change, make it compile 2020-12-09 11:13:58 +01:00
W.C.A. Wijngaards
bdfa65c6ab Import the patches from the files in the tarball in
issue #365 https://github.com/NLnetLabs/unbound/files/5659923/patches.tar.gz
from iruzanov.  The merge conflicts are fixed, but no changes are made
to the patched code.
2020-12-09 11:00:51 +01:00
W.C.A. Wijngaards
795a33c6e5 - Fix dnstap socket and the chroot not applied properly to the dnstap
socket path.
2020-10-09 08:57:23 +02:00
W.C.A. Wijngaards
48a56751e9 - Fix #305: dnstap logging significantly affects unbound performance
(regression in 1.11).
2020-09-23 11:13:52 +02:00
W.C.A. Wijngaards
45d4cbaa42 - Fix to apply chroot to dnstap-socket-path, if chroot is enabled. 2020-08-24 14:55:16 +02:00
W.C.A. Wijngaards
e856908696 dnstap io, free alloced resource on fail path. 2020-02-28 09:17:02 +01:00
W.C.A. Wijngaards
57baa78dd1 Fix memory leak in error case. From review. 2020-02-19 17:33:36 +01:00
W.C.A. Wijngaards
4d3524f496 dnstap io, test for TLS and reconnect for that. And fix unused parameters
for dt_create and fix check of socket path when using IP address.
2020-02-14 14:44:02 +01:00
W.C.A. Wijngaards
1420d59949 dnstap io, connect and write over TCP. 2020-01-31 17:07:40 +01:00
W.C.A. Wijngaards
989922631a dnstap io, fix exit when compiled without threads. 2020-01-28 15:09:21 +01:00
W.C.A. Wijngaards
874c349b44 dnstap io, remove --with-libfstrm, it is not required to build dnstap
support. protobuf-c is still used for handling the dnstap encoding.
2020-01-23 15:56:12 +01:00
W.C.A. Wijngaards
4c64c4b78e dnstap io, remove libfstrm calls from dnstap.c. 2020-01-23 15:49:36 +01:00
W.C.A. Wijngaards
9f2ac374f4 fstrm routines for start and stop, fstrm_create_control_frame_start and
fstrm_create_control_frame_stop, suitable for reuse, together with fstrm
protocol defines.
2020-01-22 11:44:11 +01:00
W.C.A. Wijngaards
efc79beb2d iothread work. 2020-01-21 17:01:25 +01:00
W.C.A. Wijngaards
57ad169605 the framestream queue. 2020-01-21 14:50:37 +01:00
Wouter Wijngaards
6bd4060ae0 portability.
git-svn-id: file:///svn/unbound/trunk@4944 be551aaa-1e26-0410-a405-d3ace91eadb9
2018-10-22 10:19:30 +00:00
Wouter Wijngaards
84a8415357 - check that the dnstap socket file can be opened and exists, print
error if not.


git-svn-id: file:///svn/unbound/trunk@4943 be551aaa-1e26-0410-a405-d3ace91eadb9
2018-10-22 10:17:38 +00:00
Wouter Wijngaards
86dcdaae33 - Fix dnstap for warning of set but not used.
git-svn-id: file:///svn/unbound/trunk@4007 be551aaa-1e26-0410-a405-d3ace91eadb9
2017-02-10 08:26:59 +00:00
Wouter Wijngaards
2958b53293 - Fix dnstap-log-resolver-response-messages, from Nikolay Edigaryev.
git-svn-id: file:///svn/unbound/trunk@3684 be551aaa-1e26-0410-a405-d3ace91eadb9
2016-03-21 09:04:21 +00:00
Wouter Wijngaards
8c2a28adbc - Fix #741: log message for dnstap socket connection is more clear.
git-svn-id: file:///svn/unbound/trunk@3612 be551aaa-1e26-0410-a405-d3ace91eadb9
2016-01-27 17:12:26 +00:00
Wouter Wijngaards
b2bdce46be - rename ldns subdirectory to sldns to avoid name collision.
git-svn-id: file:///svn/unbound/trunk@3380 be551aaa-1e26-0410-a405-d3ace91eadb9
2015-03-26 10:21:38 +00:00
Wouter Wijngaards
e3843bd0b6 remove unused variable
git-svn-id: file:///svn/unbound/trunk@3255 be551aaa-1e26-0410-a405-d3ace91eadb9
2014-11-07 09:09:37 +00:00
Wouter Wijngaards
818650efae - dnstap fixes by Robert Edmonds:
dnstap/dnstap.m4: cosmetic fixes
	dnstap/: Remove compiled protoc-c output files
	dnstap/dnstap.m4: Error out if required libraries are not found
	dnstap: Fix ProtobufCBufferSimple usage that is incorrect as of
		protobuf-c 1.0.0
	dnstap/: Adapt to API changes in latest libfstrm (>= 0.2.0)


git-svn-id: file:///svn/unbound/trunk@3254 be551aaa-1e26-0410-a405-d3ace91eadb9
2014-11-07 09:09:04 +00:00
Wouter Wijngaards
aa9a1721cc Fixup for dnstap.
git-svn-id: file:///svn/unbound/trunk@3209 be551aaa-1e26-0410-a405-d3ace91eadb9
2014-08-18 14:42:26 +00:00
Wouter Wijngaards
8ccba42b1f - dnstap support, with a patch from Farsight Security, written by
Robert Edmonds. The --enable-dnstap needs libfstrm and protobuf-c.
  It is BSD licensed (see dnstap/dnstap.c).
  Building with --enable-dnstap needs pkg-config with this patch.
- Noted dnstap in doc/README and doc/CREDITS.


git-svn-id: file:///svn/unbound/trunk@3206 be551aaa-1e26-0410-a405-d3ace91eadb9
2014-08-05 07:57:52 +00:00