Commit graph

6852 commits

Author SHA1 Message Date
Yonas Habteab
f3340ea68e
Merge 267675e80b into a810d6409b 2026-04-07 09:50:02 +02:00
Julian Brost
267675e80b RedisConnection: simplify GetOldestPendingQueryTs function 2026-04-02 16:37:57 +02:00
Yonas Habteab
7d7159c033 Reduce min queue item age from 1000ms to 300ms 2026-04-02 16:37:57 +02:00
Yonas Habteab
390ee8c02f Inline DequeueAndProcessOne & don't process items out of order
Now, the individual `ProcessQueueItem` functions decide whether to
acquire an `olock` or not instead of probing this from within the
worker loop. This is way easier than having to deal with the potential
out of order processing of items in the queue in both ways, i.e., we
don't want to send delete events for objects while their created events
haven't been processed yet and vice versa.
2026-04-02 16:37:57 +02:00
Julian Brost
855f6c7c0c IcingaDB: use key extractor for worker queue
This commit restructures the queue items so that each one now has a method
`GetQueueLookupKey()` that is used to derive which elements of the queue are
considered to be equal. For this, there is a key extractor for the
`multi_index_container` that takes the `variant` from the queue item, calls
that method on it, and puts the result in a second variant type. The types in
that variant type are automatically deduced from the return types of the
individual methods.
2026-04-02 16:37:57 +02:00
Yonas Habteab
2048450159 IcingaDB: put all queue related stuff into icingadb:task_queue namespace 2026-04-02 16:37:57 +02:00
Julian Brost
8375934d19 Simplify IcingaDB::PendingItemsThreadProc() event loop 2026-04-02 16:37:57 +02:00
Yonas Habteab
b633d6b0d0 IcingaDB: remove unused UpdateObjectAttrs method 2026-04-02 16:37:57 +02:00
Yonas Habteab
89d8c326e6 Fix missing olock for dependency child registration
Previously, the checkable was locked while processing all the dependency
registration stuff, so the worker thread should also do the same to
avoid any potential race conditions.
2026-04-02 16:37:57 +02:00
Julian Brost
9d5883df78 IcingaDB: use polymorphism for queue entries 2026-04-02 16:37:57 +02:00
Yonas Habteab
e88366ddae IcingaDB: subscribe to OnNextCheckChanged signal
We can't drop the `OnNextCheckUpdated` signal entirely yet, as IDO still
relies on it.
2026-04-02 16:37:57 +02:00
Yonas Habteab
485227390f Revert "CheckerComponent#CheckThreadProc(): also propagate next check update to Icinga DB"
This reverts commit e9b8c67975.
2026-04-02 16:37:57 +02:00
Yonas Habteab
bbb7d0249e RedisConnection: enhance WriteQueueItem & related usages 2026-04-02 16:37:57 +02:00
Yonas Habteab
cbb4147055 RedisConnection: simplify query prioritization logic
As opposed to the previous version which used a complex data structure
to correctly manage the query priorities, this version uses two separate
queues for the high and normal priority writes. All high priority writes
are processed in FIFO order but over take all queries from the normal
priority queue. The later queue only be processed when the high priority
queue is empty.
2026-04-02 16:37:57 +02:00
Yonas Habteab
adbefa5540 Revert "IcingaDB: suppress state sync until config sync finished"
This reverts commit f6f7d9b635 and all
other its new users.
2026-04-02 14:06:28 +02:00
Yonas Habteab
d364ad981e IcingaDB: enqueue config runtime updates to the worker queue 2026-04-02 14:06:28 +02:00
Yonas Habteab
4dbf782e4e OTel: raise runtime error when failing to fully serialize Protobuf request 2026-04-02 10:51:35 +02:00
Yonas Habteab
465650262a OTel: add connect & handshake timeout 2026-04-02 10:51:35 +02:00
Julian Brost
1139ba9b0d OTel: replace AsioDualEvent usage with AsioConditionVariable 2026-04-02 10:51:35 +02:00
Yonas Habteab
044f85ee76 OTel: do not perform graceful disconnect on I/O timeout 2026-04-01 12:18:22 +02:00
Yonas Habteab
96c3364ab0 OTel: fix race condition triggered on Icinga 2 reload/shutdown
Co-Authored-By: Julian Brost <julian.brost@icinga.com>
2026-04-01 12:18:22 +02:00
Yonas Habteab
715aacc19c Don't manually include custom Protobuf dir via compiler flag
Co-Authored-By: Johannes Schmidt <johannes.schmidt@icinga.com>
2026-04-01 12:18:21 +02:00
Yonas Habteab
e6c420e106 OTLP: Set enable_ha to true by default 2026-04-01 12:18:21 +02:00
Yonas Habteab
3f68eea1fd Reduce default flush_threshold to 16MiB
So that it doesn't cause `request body too large` errors when used with
the default OpenTelemetry Collector config that has `max_request_body_size`
set to `20MiB`.
2026-04-01 12:18:21 +02:00
Julian Brost
8f36bdcddc Replace for with a simpler while loop & fix a typo 2026-04-01 12:18:21 +02:00
Yonas Habteab
8bdfba8772 Allow users to provide additional resource attributes 2026-04-01 12:18:21 +02:00
Yonas Habteab
60fe45cd6e Add OTLPMetricsWriter 2026-04-01 12:18:21 +02:00
Yonas Habteab
415140bc36 Add common OTel type/lib 2026-04-01 12:18:21 +02:00
Yonas Habteab
374cc6e282 Cache Icinga DB env_id in Application class as well
So that other components can use it without having to import any Icinga
DB related header files, but only the base library.
2026-04-01 12:15:58 +02:00
Julian Brost
4f13651cb0
Merge pull request #10727 from Icinga/icingadb-missing-exception-messages
Redis exceptions: add proper what() messages
2026-03-31 10:27:33 +02:00
Julian Brost
9d361e1fb3
Merge pull request #10734 from Icinga/deprecate-everything-we-dont-like
Schedule deprecated features for removal in v2.18
2026-03-31 10:25:44 +02:00
Julian Brost
207764584a Redis exceptions: remove inline specifiers
Remove them as they are redundant, as requested in the PR review.
2026-03-27 17:02:05 +01:00
Julian Brost
221382486e Redis exceptions: use BOOST_THROW_EXCEPTION
Use it for consistency, as requested in the PR review.
2026-03-27 17:01:51 +01:00
Julian Brost
862f012381 Redis exceptions: add proper what() messages
RedisDisconnected::what() and RedisProtocolError::what() always returned an
empty string. Similarly, BadRedisType::what() and BadRedisInt::what() only
return the value that couldn't be parsed without any information about the
exception type. If only what() is used when printing the exception, as it's
typical, this results in unhelpful log messages like the following when simply
stopping the Redis server:

    [2026-02-23 14:33:33 +0100] critical/IcingaDB: Error during receiving the response to a query which has been fired and forgotten: Connection reset by peer [system:104 at /usr/include/boost/asio/detail/reactive_socket_recv_op.hpp:134 in function 'do_complete']
    [2026-02-23 14:33:33 +0100] critical/IcingaDB: Error during receiving the response to a query which has been fired and forgotten:
    [2026-02-23 14:33:33 +0100] critical/IcingaDB: Error during receiving the response to a query which has been fired and forgotten:
    [2026-02-23 14:33:33 +0100] critical/IcingaDB: Cannot connect to redis-1:6379: Connection refused [system:111 at /usr/include/boost/asio/detail/reactive_socket_connect_op.hpp:98 in function 'do_complete']

This commit changes these messages so that something like "Redis disconnected",
"Redis protocol error: bad int: foo", or "Redis protocol error: bad type: ?" is
returned. In doing so, it also removes a member of type std::vector<char> in
BadRedisInt as this is unsafe to use in exceptions (it violates the requirement
that copy constructor and assignment must be nothrow, see
https://en.cppreference.com/w/cpp/error/exception.html#Standard_exception_requirements).

With this commit, the log messages are now a bit more helpful:

    [2026-02-23 15:08:23 +0100] critical/IcingaDB: Error during receiving the response to a query which has been fired and forgotten: Connection reset by peer [system:104 at /usr/include/boost/asio/detail/reactive_socket_recv_op.hpp:134 in function 'do_complete']
    [2026-02-23 15:08:23 +0100] critical/IcingaDB: Error during receiving the response to a query which has been fired and forgotten: Redis disconnected
    [2026-02-23 15:08:23 +0100] critical/IcingaDB: Error during receiving the response to a query which has been fired and forgotten: Redis disconnected
    [2026-02-23 15:08:23 +0100] critical/IcingaDB: Cannot connect to redis-1:6379: Connection refused [system:111 at /usr/include/boost/asio/detail/reactive_socket_connect_op.hpp:98 in function 'do_complete']
2026-03-27 17:01:49 +01:00
Johannes Schmidt
2108300cf8 Add warnings to deprecated features indicating removal in v2.18 2026-03-27 14:20:55 +01:00
Yonas Habteab
895dea2dc4
Merge pull request #10770 from Icinga/problem-chars
Some checks failed
Container Image / Container Image (push) Has been cancelled
Linux / alpine:bash (push) Has been cancelled
Linux / amazonlinux:2 (push) Has been cancelled
Linux / amazonlinux:2023 (push) Has been cancelled
Linux / debian:11 (linux/386) (push) Has been cancelled
Linux / debian:11 (push) Has been cancelled
Linux / debian:12 (linux/386) (push) Has been cancelled
Linux / debian:12 (push) Has been cancelled
Linux / debian:13 (push) Has been cancelled
Linux / fedora:41 (push) Has been cancelled
Linux / fedora:42 (push) Has been cancelled
Linux / fedora:43 (push) Has been cancelled
Linux / opensuse/leap:15.6 (push) Has been cancelled
Linux / opensuse/leap:16.0 (push) Has been cancelled
Linux / registry.suse.com/bci/bci-base:16.0 (push) Has been cancelled
Linux / registry.suse.com/suse/sle15:15.6 (push) Has been cancelled
Linux / registry.suse.com/suse/sle15:15.7 (push) Has been cancelled
Linux / rockylinux/rockylinux:10 (push) Has been cancelled
Linux / rockylinux:8 (push) Has been cancelled
Linux / rockylinux:9 (push) Has been cancelled
Linux / ubuntu:22.04 (push) Has been cancelled
Linux / ubuntu:24.04 (push) Has been cancelled
Linux / ubuntu:25.04 (push) Has been cancelled
Linux / ubuntu:25.10 (push) Has been cancelled
Windows / Windows (push) Has been cancelled
Warn on problematic object names
2026-03-27 10:21:58 +01:00
Yonas Habteab
4039b39061 Warn if object name contains ctrl characters 2026-03-26 15:01:15 +01:00
Yonas Habteab
da521203b1 Endpoint: warn if endpoint name exceeds 64 characters 2026-03-26 13:58:57 +01:00
Yonas Habteab
5a4cfae182 Warn of object name exceeds 255 characters 2026-03-26 13:58:57 +01:00
Yonas Habteab
2e6d23701a Warn about leading and trailing WSP in object names 2026-03-26 13:58:57 +01:00
Julian Brost
ed403294a3 OutgoingHttpMessage: don't use shared_ptr for m_CpuBoundWork
This change just gives clear ownership over the CpuBoundWork to the
OutgoingHttpMessage, instead of the previous shared_ptr and weak_ptr
combination with an unclear purpose.
2026-03-26 11:47:45 +01:00
Johannes Schmidt
7d4af75189 Make ShowCodeLocation() show the full line with a LastColumn of 0 2026-03-24 13:53:31 +01:00
Yonas Habteab
26d23bd569 IcingaDB: enqueue all dependency related updates to the worker queue 2026-03-23 11:26:48 +01:00
Yonas Habteab
e6dc594313 IcingaDB: don't send nextupdate:{host,service} as part of the initial sync
Don't send nextupdate as part of the intial sync. Instead, enque them to
the background worker to be sent after the initial dump is done.
2026-03-23 11:26:48 +01:00
Yonas Habteab
eab507f892 IcingaDB: track object changesets efficiently 2026-03-23 11:26:48 +01:00
Yonas Habteab
105c066649 IcingaDB: use a separate Redis conn for history & heartbeat streams 2026-03-23 11:26:47 +01:00
Alexander A. Klimov
85b16d2186 Remove unused IoEngine#m_AlreadyExpiredTimer 2026-03-19 14:53:29 +01:00
Alexander A. Klimov
7b45d8d625 Remove unused IoEngine::YieldCurrentCoroutine() 2026-03-19 14:53:29 +01:00
Alexander A. Klimov
cefa942c31 Make IoEngine#m_CpuBoundSemaphore unsigned 2026-03-19 14:53:29 +01:00
Alexander A. Klimov
06cfdbb650 CpuBoundWork#CpuBoundWork(): don't spin on atomic int to acquire slot
This is inefficient and involves possible bad surprises regarding
waiting durations on busy nodes. Instead, use `AsioConditionVariable#Wait()`
if there are no free slots. It's notified by others'
`CpuBoundWork#~CpuBoundWork()` once finished.
2026-03-19 14:53:29 +01:00