This fixes a deadlock in the GraphiteWriter feature, which is visible
during the reload process. The reload thread waits for the
GraphiteWriter to finish, but the GraphiteWriter can't finish because
it's stuck in `SendMetric()` waiting for a lock which is hold by the
reload thread.
This patch ensures that specific configuration types
are pre-activated and post-activated. In general,
logging is first, then common configuration objects
like host/service, downtimes, etc.
In the end, all features are activated after to ensure
that notifications are only sent once downtimes are applied.
A similar thing happens for starting with checks too early.
The ApiListener feature runs first to allow cluster connections
at first glance.
fixes#6057fixes#6231
What does this change?
* Remove use of spaces for formatting
These could be found by using `grep -r -l -P '^\t+ +[^*]'
* Removal of training whitespaces
* A few lines longer than 120 chars
This better reflects its purpose as otherwise it would imply
that you need Elastic Stack for it. Graylog also reads from
Elasticsearch instances, this could serve as additional integration
here.
Rather than leaving stale connections about we tried to poll for data coming in
from InfluxDB and timeout if it didn't repond in a timely manner. This introduced
a race where the timeout triggers, a context switch occurs where data is actually
available and the TlsStream spins trying to asynchronously notify that data is
available, but which never gets read. Not only does this use up 100% of a core,
but it also slowly starves the system of handler threads at which point metrics
stop being delivered.
This basically removes the poll and timeout, any TLS socket erros should be
detected by TCP keep-alives.
Fixes#5460#5469
On a non 204 response we parse the HTTP response until complete e.g. do the headers
and body, not just the header. A new interface is added to the response to allow us
to determine the body size so that it may be read out and buffered. The body is
parsed and any error message printed out. In the event that the parsing fails the
raw body is dumped out; better than nothing!
fixes#4411
Signed-off-by: Michael Friedrich <michael.friedrich@icinga.com>
Exposes the TCP socket used to communicate with the InfluxDB server. When we are
expecing a response we can now call poll() on the socket to wait for data to become
available. If it doesn't in a user configurable timeout period we abort the request.
fixes#4927fixes#4941
Signed-off-by: Michael Friedrich <michael.friedrich@icinga.com>
Previously the logic would just bail out if no performance data was associated with a
check, the problem being that check metadata was skipped too. This rearranges the code
to dump out performance metrics if they exist, then dump out metadata if requested. This
also fixes an issue whereby metadata was being sent for every performance data in the
check result, rather than just once, so we save a bit of bandwidth as a result.
fixes#12276
Signed-off-by: Michael Friedrich <michael.friedrich@icinga.com>
Backslashes escape spaces or commas (and evidently equals), given tags are
separated by commas, tag keys and values are separated by equals and tags
are separated from fields by a space we need to take action when these end
in a backslash e.g. 'C:\'. Also discovered a bug whereby the metric tag was
missing out on escaping.
fixes#12227
Signed-off-by: Gunnar Beutner <gunnar.beutner@netways.de>
The escaping wasn't being performed on measuerments, keys or tag values. The
escape function was returning the input and not the modified ouput, so this
has been fixed
refs #12047
Signed-off-by: Michael Friedrich <michael.friedrich@netways.de>
Adds a new configuration variable in keeping with the graphite writer
which defaults to false to save network bandwidth. All metrics currently
supported by graphite are now available to InfluxDB. I added in some
formatting functions, to handle integers and booleans as we know and
control their types, and the supporting regexes in the sanity checker.
Updating to InfluxDB 0.13.X started giving 400 errors due to the missing
Host header in HTTP/1.1 requests. HttpRequest has been updated to auto-
magically add the host and port to these requests if not explicitly
stated by the client code.
The exception code has been cleaned up to break out of the function
early if such a condition is raised, this avoids unnecessarily executing
code which will ultimately fail.
fixes#11912
Signed-off-by: Gunnar Beutner <gunnar.beutner@netways.de>
Fixes a couple issues to do with line formatting of influx DB data points. All
keys and values need commas and white space escaping. Values are also checked
for type. If a numeric or scientific value is detected this is output as an
Influx floating point/scientific number. Booleans are detected and output in
a canonical format. All other values are strings, which have double quotes
escaped and the entire string is wrapped in double quotes. The handling of
thresholds has changed before this becomes officially released. These values
if available are passed to the accumulation function in a dictionary, said
dictionary builds a single data point with multiple fields, rather than the
existing 5 data points, thus saving bandwidth costs.
fixes#11904
Signed-off-by: Gunnar Beutner <gunnar.beutner@netways.de>
Adds an Icinga2 object to directly interface with InfluxDB's native HTTP API.
This supports optional basic authorization, and TLS transport. InfluxDB didn't
appear to like having the TLS stream kept open, so instead this object buffers
data points which are then flushed to InfluxDB as a batch write, either driven
by a configurable timeout or threshold.
As InfluxDB is a schema-less database the host and service templates are user
configurable allowing both the measurement field and tags to be set by the
end user via macro expansion. This allows access to tag fields from arbitrary
data associated with host.vars or service.vars. If a particular value is
unable to be resolved, the tag will be dropped and not transmitted to InfluxDB.
Also alters URL handling to omit array brackets when only a single value is
attached to a key, otherwise InfluxDB has a strop with non-standard syntax.
fixes#10480
Signed-off-by: Michael Friedrich <michael.friedrich@netways.de>
This changes the entire tree, but with the prefix "icinga2"
not to conflict with existing installations. Includes
enable_legacy_mode and detailed documentation.
fixes#9461fixes#8149
