From 283f68cad209a3fd9409246efec9c53f82e216cd Mon Sep 17 00:00:00 2001 From: Filip Siroky Date: Mon, 7 Nov 2016 09:33:13 +0100 Subject: [PATCH] doc: add statistics facility --- doc/introduction.rst | 1 + doc/man/knot.conf.5in | 279 ++++++++++++++++++++++++++++++++++++++++++ doc/man/knotc.8in | 8 ++ doc/man_knotc.rst | 8 ++ doc/modules.rst | 30 +++++ doc/operation.rst | 21 ++++ doc/reference.rst | 254 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 601 insertions(+) diff --git a/doc/introduction.rst b/doc/introduction.rst index 5243280c5..1e80feab8 100644 --- a/doc/introduction.rst +++ b/doc/introduction.rst @@ -48,6 +48,7 @@ Server features: * Automatic DNSSEC signing * PKCS #11 interface * Forward and reverse records synthesis +* DNS request traffic statistics For more info and downloads see `www.knot-dns.cz `_. diff --git a/doc/man/knot.conf.5in b/doc/man/knot.conf.5in index 8c78108f1..f03d0c5d7 100644 --- a/doc/man/knot.conf.5in +++ b/doc/man/knot.conf.5in @@ -440,6 +440,39 @@ A UNIX socket path where the server listens for control commands. Maximum time the control socket operations can take. Set 0 for infinity. .sp \fIDefault:\fP 5 +.SH STATISTICS SECTION +.sp +Periodic server statistics dumping. +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +statistics: + timer: TIME + file: STR + append: BOOL +.ft P +.fi +.UNINDENT +.UNINDENT +.SS timer +.sp +A period after which all available statistics metrics will by written to the +\fI\%file\fP\&. +.sp +\fIDefault:\fP not set +.SS file +.sp +A file path of statistics output in the YAML format. +.sp +\fIDefault:\fP \fI\%rundir\fP/stats.yaml +.SS append +.sp +If enabled, the output will be appended to the \fI\%file\fP +instead of file replacement. +.sp +\fIDefault:\fP off .SH KEYSTORE SECTION .sp DNSSEC keystore configuration. @@ -1233,6 +1266,252 @@ A module identifier. A path to the directory where the database is stored. .sp \fIRequired\fP +.SH MODULE STATS +.sp +The module provides incoming query processing statistics. +.sp +\fBNOTE:\fP +.INDENT 0.0 +.INDENT 3.5 +Leading 16\-bit message size over TCP is not considered. +.UNINDENT +.UNINDENT +.INDENT 0.0 +.INDENT 3.5 +.sp +.nf +.ft C +mod\-stats: + \- id: STR + request\-protocol: BOOL + server\-operation: BOOL + request\-bytes: BOOL + response\-bytes: BOOL + edns\-presence: BOOL + flag\-presence: BOOL + response\-code: BOOL + reply\-nodata: BOOL + query\-type: BOOL + query\-size: BOOL + reply\-size: BOOL +.ft P +.fi +.UNINDENT +.UNINDENT +.SS id +.sp +A module identifier. +.SS request\-protocol +.sp +If enabled, all incoming requests are counted by the network protocol: +.INDENT 0.0 +.IP \(bu 2 +udp4 \- UDP over IPv4 +.IP \(bu 2 +tcp4 \- TCP over IPv4 +.IP \(bu 2 +udp6 \- UDP over IPv6 +.IP \(bu 2 +tcp6 \- TCP over IPv6 +.UNINDENT +.sp +\fIDefault:\fP on +.SS server\-operation +.sp +If enabled, all incoming requests are counted by the server operation. The +server operation is based on message header OpCode and message query (meta) type: +.INDENT 0.0 +.IP \(bu 2 +query \- Normal query operation +.IP \(bu 2 +update \- Dynamic update operation +.IP \(bu 2 +notify \- NOTIFY request operation +.IP \(bu 2 +axfr \- Full zone transfer operation +.IP \(bu 2 +ixfr \- Incremental zone transfer operation +.IP \(bu 2 +invalid \- Invalid server operation +.UNINDENT +.sp +\fIDefault:\fP on +.SS request\-bytes +.sp +If enabled, all incoming request bytes are counted by the server operation: +.INDENT 0.0 +.IP \(bu 2 +query \- Normal query bytes +.IP \(bu 2 +update \- Dynamic update bytes +.IP \(bu 2 +other \- Other request bytes +.UNINDENT +.sp +\fIDefault:\fP on +.SS response\-bytes +.sp +If enabled, outgoing response bytes are counted by the server operation: +.INDENT 0.0 +.IP \(bu 2 +reply \- Normal response bytes +.IP \(bu 2 +transfer \- Zone transfer bytes +.IP \(bu 2 +other \- Other response bytes +.UNINDENT +.sp +\fBWARNING:\fP +.INDENT 0.0 +.INDENT 3.5 +Dynamic update response bytes are not counted by this module. +.UNINDENT +.UNINDENT +.sp +\fIDefault:\fP on +.SS edns\-presence +.sp +If enabled, EDNS pseudo section presence is counted by the message direction: +.INDENT 0.0 +.IP \(bu 2 +request \- EDNS present in request +.IP \(bu 2 +response \- EDNS present in response +.UNINDENT +.sp +\fIDefault:\fP off +.SS flag\-presence +.sp +If enabled, some message header flags are counted: +.INDENT 0.0 +.IP \(bu 2 +TC \- Truncated Answer in response +.IP \(bu 2 +DO \- DNSSEC OK in request +.UNINDENT +.sp +\fIDefault:\fP off +.SS response\-code +.sp +If enabled, outgoing response code is counted: +.INDENT 0.0 +.IP \(bu 2 +NOERROR +.IP \(bu 2 +\&... +.IP \(bu 2 +NOTZONE +.IP \(bu 2 +BADVERS +.IP \(bu 2 +\&... +.IP \(bu 2 +BADCOOKIE +.IP \(bu 2 +other \- All other codes +.UNINDENT +.sp +\fBNOTE:\fP +.INDENT 0.0 +.INDENT 3.5 +In the case of multi\-message zone transfer response, just one counter is +incremented. +.UNINDENT +.UNINDENT +.sp +\fBWARNING:\fP +.INDENT 0.0 +.INDENT 3.5 +Dynamic update response code is not counted by this module. +.UNINDENT +.UNINDENT +.sp +\fIDefault:\fP on +.SS reply\-nodata +.sp +If enabled, NODATA pseudo RCODE (see RFC 2308, Section 2.2) is counted by the +query type: +.INDENT 0.0 +.IP \(bu 2 +A +.IP \(bu 2 +AAAA +.IP \(bu 2 +other \- All other types +.UNINDENT +.sp +\fIDefault:\fP off +.SS query\-type +.sp +If enabled, normal query type is counted: +.INDENT 0.0 +.IP \(bu 2 +A (TYPE1) +.IP \(bu 2 +\&... +.IP \(bu 2 +TYPE65 +.IP \(bu 2 +SPF (TYPE99) +.IP \(bu 2 +\&... +.IP \(bu 2 +TYPE110 +.IP \(bu 2 +ANY (TYPE255) +.IP \(bu 2 +\&... +.IP \(bu 2 +TYPE260 +.IP \(bu 2 +other \- All other types +.UNINDENT +.sp +\fBNOTE:\fP +.INDENT 0.0 +.INDENT 3.5 +Not all assigned meta types (IXFR, AXFR,...) have their own counters, +because such types are not processed as normal query. +.UNINDENT +.UNINDENT +.sp +\fIDefault:\fP off +.SS query\-size +.sp +If enabled, normal query message size distribution is counted by the size range +in bytes: +.INDENT 0.0 +.IP \(bu 2 +0\-15 +.IP \(bu 2 +16\-31 +.IP \(bu 2 +\&... +.IP \(bu 2 +272\-287 +.IP \(bu 2 +288\-65535 +.UNINDENT +.sp +\fIDefault:\fP off +.SS reply\-size +.sp +If enabled, normal reply message size distribution is counted by the size range +in bytes: +.INDENT 0.0 +.IP \(bu 2 +0\-15 +.IP \(bu 2 +16\-31 +.IP \(bu 2 +\&... +.IP \(bu 2 +4080\-4095 +.IP \(bu 2 +4096\-65535 +.UNINDENT +.sp +\fIDefault:\fP off .SH AUTHOR CZ.NIC Labs .SH COPYRIGHT diff --git a/doc/man/knotc.8in b/doc/man/knotc.8in index ccf42cf50..c0b99dea5 100644 --- a/doc/man/knotc.8in +++ b/doc/man/knotc.8in @@ -78,6 +78,10 @@ Stop the server if running. Reload the server configuration and modified zone files. All open zone transactions will be aborted! .TP +\fBstats\fP [\fImodule\fP[\fB\&.\fP\fIcounter\fP]] +Show global statistics counter(s). To print also counters with value 0, use +force option. +.TP \fBzone\-check\fP [\fIzone\fP\&...] Test if the server can load the zone. Semantic checks are executed if enabled in the configuration. (*) @@ -137,6 +141,10 @@ Remove zone data within the transaction. \fBzone\-purge\fP \fIzone\fP\&... Purge zone data, file, journal, and timers. .TP +\fBzone\-stats\fP \fIzone\fP [\fImodule\fP[\fB\&.\fP\fIcounter\fP]] +Show zone statistics counter(s). To print also counters with value 0, use +force option. +.TP \fBconf\-init\fP Initialize the configuration database. (*) .TP diff --git a/doc/man_knotc.rst b/doc/man_knotc.rst index 75762c4d1..845b76f43 100644 --- a/doc/man_knotc.rst +++ b/doc/man_knotc.rst @@ -55,6 +55,10 @@ Actions Reload the server configuration and modified zone files. All open zone transactions will be aborted! +**stats** [*module*\ [\ **.**\ *counter*\ ]] + Show global statistics counter(s). To print also counters with value 0, use + force option. + **zone-check** [*zone*...] Test if the server can load the zone. Semantic checks are executed if enabled in the configuration. (*) @@ -114,6 +118,10 @@ Actions **zone-purge** *zone*... Purge zone data, file, journal, and timers. +**zone-stats** *zone* [*module*\ [\ **.**\ *counter*\ ]] + Show zone statistics counter(s). To print also counters with value 0, use + force option. + **conf-init** Initialize the configuration database. (*) diff --git a/doc/modules.rst b/doc/modules.rst index adbac0399..1fca0c848 100644 --- a/doc/modules.rst +++ b/doc/modules.rst @@ -515,3 +515,33 @@ to the configuration file:: .. NOTE:: This module is not configurable. + +``stats`` — query statistics +---------------------------- + +The module extends server statistics with incoming DNS request and corresponding +response counters, such as used network protocol, total number of responded bytes, +etc (see :ref:`mod-stats` for full list of supported counters). +This module should be configured as the last module. + +Common statistics with default module configuration:: + + template: + - id: default + global-module: mod-stats + +Per zone statistics with explicit module configuration:: + + mod-stats: + - id: custom + edns-presence: on + query-type: on + + template: + - id: default + module: mod-stats/custom + +.. NOTE:: + Server initiated communication (outgoing NOTIFY, incoming \*XFR,...) is not + counted by this module. + diff --git a/doc/operation.rst b/doc/operation.rst index 47c72c387..6d94fe68f 100644 --- a/doc/operation.rst +++ b/doc/operation.rst @@ -266,3 +266,24 @@ no differences will be created. If you want to refresh the slave zones, you can do this with:: $ knotc zone-refresh + +.. _Statistics: + +Statistics +========== + +The server provides some general statistics and optional query module statistics +(see :ref:`mod-stats`). + +Server statistics or global module statistics can be shown by:: + + $ knotc stats + $ knotc stats server # Show all server counters + $ knotc stats mod-stats # Show all mod-stats counters + $ knotc stats server.zone-count # Show specific server counter + +Per zone statistics can be shown by:: + + $ knotc zone-stats example.com mod-stats + +To show all supported counters even with 0 value use the force option. diff --git a/doc/reference.rst b/doc/reference.rst index 59662e6cf..783c421b6 100644 --- a/doc/reference.rst +++ b/doc/reference.rst @@ -506,6 +506,49 @@ Maximum time the control socket operations can take. Set 0 for infinity. *Default:* 5 +.. _statistics_section: + +Statistics section +================== + +Periodic server statistics dumping. + +:: + + statistics: + timer: TIME + file: STR + append: BOOL + +.. _statistics_timer: + +timer +----- + +A period after which all available statistics metrics will by written to the +:ref:`file`. + +*Default:* not set + +.. _statistics_file: + +file +---- + +A file path of statistics output in the YAML format. + +*Default:* :ref:`rundir`/stats.yaml + +.. _statistics_append: + +append +------ + +If enabled, the output will be appended to the :ref:`file` +instead of file replacement. + +*Default:* off + .. _Keystore section: Keystore section @@ -1464,3 +1507,214 @@ dbdir A path to the directory where the database is stored. *Required* + +.. _mod-stats: + +Module stats +============ + +The module provides incoming query processing statistics. + +.. NOTE:: + Leading 16-bit message size over TCP is not considered. + +:: + + mod-stats: + - id: STR + request-protocol: BOOL + server-operation: BOOL + request-bytes: BOOL + response-bytes: BOOL + edns-presence: BOOL + flag-presence: BOOL + response-code: BOOL + reply-nodata: BOOL + query-type: BOOL + query-size: BOOL + reply-size: BOOL + +.. _mod-stats_id: + +id +-- + +A module identifier. + +.. _mod-stats_request-protocol: + +request-protocol +---------------- + +If enabled, all incoming requests are counted by the network protocol: + +* udp4 - UDP over IPv4 +* tcp4 - TCP over IPv4 +* udp6 - UDP over IPv6 +* tcp6 - TCP over IPv6 + +*Default:* on + +.. _mod-stats_server-operation: + +server-operation +---------------- + +If enabled, all incoming requests are counted by the server operation. The +server operation is based on message header OpCode and message query (meta) type: + +* query - Normal query operation +* update - Dynamic update operation +* notify - NOTIFY request operation +* axfr - Full zone transfer operation +* ixfr - Incremental zone transfer operation +* invalid - Invalid server operation + +*Default:* on + +.. _mod-stats_request-bytes: + +request-bytes +------------- + +If enabled, all incoming request bytes are counted by the server operation: + +* query - Normal query bytes +* update - Dynamic update bytes +* other - Other request bytes + +*Default:* on + +.. _mod-stats_response-bytes: + +response-bytes +-------------- + +If enabled, outgoing response bytes are counted by the server operation: + +* reply - Normal response bytes +* transfer - Zone transfer bytes +* other - Other response bytes + +.. WARNING:: + Dynamic update response bytes are not counted by this module. + +*Default:* on + +.. _mod-stats_edns-presence: + +edns-presence +------------- + +If enabled, EDNS pseudo section presence is counted by the message direction: + +* request - EDNS present in request +* response - EDNS present in response + +*Default:* off + +.. _mod-stats_flag-presence: + +flag-presence +------------- + +If enabled, some message header flags are counted: + +* TC - Truncated Answer in response +* DO - DNSSEC OK in request + +*Default:* off + +.. _mod-stats_response-code: + +response-code +------------- + +If enabled, outgoing response code is counted: + +* NOERROR +* ... +* NOTZONE +* BADVERS +* ... +* BADCOOKIE +* other - All other codes + +.. NOTE:: + In the case of multi-message zone transfer response, just one counter is + incremented. + +.. WARNING:: + Dynamic update response code is not counted by this module. + +*Default:* on + +.. _mod-stats_reply-nodata: + +reply-nodata +------------ + +If enabled, NODATA pseudo RCODE (see RFC 2308, Section 2.2) is counted by the +query type: + +* A +* AAAA +* other - All other types + +*Default:* off + +.. _mod-stats_query-type: + +query-type +---------- + +If enabled, normal query type is counted: + +* A (TYPE1) +* ... +* TYPE65 +* SPF (TYPE99) +* ... +* TYPE110 +* ANY (TYPE255) +* ... +* TYPE260 +* other - All other types + +.. NOTE:: + Not all assigned meta types (IXFR, AXFR,...) have their own counters, + because such types are not processed as normal query. + +*Default:* off + +.. _mod-stats_query-size: + +query-size +---------- + +If enabled, normal query message size distribution is counted by the size range +in bytes: + +* 0-15 +* 16-31 +* ... +* 272-287 +* 288-65535 + +*Default:* off + +.. _mod-stats_reply-size: + +reply-size +---------- + +If enabled, normal reply message size distribution is counted by the size range +in bytes: + +* 0-15 +* 16-31 +* ... +* 4080-4095 +* 4096-65535 + +*Default:* off