upstream/icingadb
1
0
Fork 0
mirror of https://github.com/Icinga/icingadb.git synced 2026-04-21 06:07:19 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 7
1971 commits 28 branches 11 tags 14 MiB
Commit graph

602 commits

Author SHA1 Message Date
Alexander A. Klimov
48b7bb4c35 DB#Delete(): allow monitoring succeeded items 2022-06-15 09:51:59 +02:00
Alexander A. Klimov
cc352252ec DB#CreateStreamed(): allow monitoring succeeded items 2022-06-15 09:51:59 +02:00
Alexander A. Klimov
db4725218e DB#{*BulkExec,*Streamed}(): replace succeeded channel with callbacks 
to allow to avoid a goroutine reading that channel.
 2022-06-15 09:51:59 +02:00
Julian Brost
1c02f89b9b
Merge pull request #493 from Icinga/schema-consistency-for-relations 
Schema consistency for relations
 2022-06-09 10:19:46 +02:00
Yonas Habteab
154f46c2f0 Checkable: Adjust the naming of the struct/json keys according to the new scheme 2022-05-31 15:15:29 +02:00
Julian Brost
cd78d96f04 Fix icingadb --version output when using older Git versions 
When building Icinga DB from a tarball generated using `git archive` using a
Git version older than 2.32, the version output could incorrectly state "Icinga
DB version: %(describe)". This commit detects this situation and implements a
fallback.
 2022-05-24 17:53:19 +02:00
Julian Brost
64aca11bcb
Merge pull request #489 from Icinga/retry-dns-errors 
Streamline IsRetryable functions and retry DNS errors
 2022-05-24 16:43:54 +02:00
Eric Lippmann
51068fff46
Merge pull request #487 from Icinga/feature/version-details 
Add more information to `icingadb --version` ouput
 2022-05-24 16:04:38 +02:00
Eric Lippmann
53cee3ee04 Streamline IsRetryable functions and retry DNS errors 
Aligns the behavior of the IsRetryable functions for Redis and database.
It now also checks if the error implements timeout, even if the error
implements temporary, where previously only the temporary bool was
returned and timeout was ignored. The idea here is that an error could
implement both, returning false for temporary and true for timeout.
Also, all DNS errors are now considered retryable.
 2022-05-24 16:02:34 +02:00
Julian Brost
c6e843146f Add more information to icingadb --version ouput 
This commit adds the build information (Go version, Git commit) and system
information (distribution with version) to the output of `icingadb --version`.

When exporting the source code using `git archive`, Git will insert the version
information using the export-subst attribute (adds both a pretty version string
like `git describe` and the commit hash).

`go build` also adds some Git commit information to the resulting binary when
building from a Git working directory (only the commit hash and a flag if there
are uncommitted changes, but no pretty version string).

Note that `go build` does not include Git version information in the binary, so
when running directly from a Git working directory, commit information won't be
available in the binary, but when doing this, you should be well aware which
version you're running anyways.

System information is obtained from the `os-release` file.
 2022-05-23 16:03:56 +02:00
Julian Brost
5f5b3fb919
Merge pull request #482 from Icinga/feature/parameter-delimiters-check-execution-6277 
Add CommandArgument#Separator
 2022-05-23 13:23:51 +02:00
Alexander A. Klimov
c1c15331e4 Quote column names for the case they're reserved 2022-05-17 17:57:55 +02:00
Alexander A. Klimov
1cacfbe7ba Add CommandArgument#Separator 2022-05-16 17:26:25 +02:00
Eric Lippmann
5e3a85976d Expect serverName only in TLS.MakeConfig() 
With recent changes to Unix domain socket support, port and address
handling in SQL and Redis configuration, the function can now be reduced
to just expecting the server name.
 2022-05-16 17:01:09 +02:00
Eric Lippmann
399b6a3d32 Split Redis address into host and port 
The port is set to 6379 by default to keep it consistent with standard
Redis installations, but the example configuration explicitly specifies
6380 since that is the default port of the icingadb-redis package.
 2022-05-16 17:01:09 +02:00
Julian Brost
eccac78ff3 Retention: take environment_id into account 
This commit ensures that the retention feature only deletes history from the
current environment. Previously, the deletion queries where missing an
appropriate WHERE clause.
 2022-05-13 17:19:03 +02:00
Julian Brost
da3fa88707
Merge pull request #477 from Icinga/feature/more-config-validation 
Check for required database and Redis config options during config validation
 2022-05-13 16:09:00 +02:00
Julian Brost
21bc728783 CleanupOlderThan: no longer return number of rounds 
This value is not used anymore, so instead of returning a struct, the function
can just return the number of rows deleted as uint64.
 2022-05-13 15:30:06 +02:00
Julian Brost
5b6c37feca Retention: remove StartWithCallback 
This was added for the tests but the corresponding test was rewritten in #467
so it's no longer used or needed.
 2022-05-13 15:30:06 +02:00
Julian Brost
3335595587 Build fix 
Two PRs (#247, #480) were merged without hard merge conflicts but the resulting
code missed a change of a variable leading to a build failure.
 2022-05-13 15:27:25 +02:00
Julian Brost
772e9bddfd
Merge pull request #480 from Icinga/feature/retention-logging 
Retention: periodically log number of items removed
 2022-05-13 15:17:18 +02:00
Julian Brost
5081ded974 Check for required database and Redis config options during config validation 2022-05-13 15:09:31 +02:00
Eric Lippmann
8480ba949a
Merge pull request #483 from Icinga/bugfix/sql-port-numbers 
DB: use default ports and allow setting port for PostgreSQL Unix domain sockets
 2022-05-13 13:41:16 +02:00
Julian Brost
19170ecbcf SLA reporting: history retention for SLA tables 2022-05-13 12:04:47 +02:00
Julian Brost
e9bd5cd40f SLA reporting: additionally write relevant history events to dedicated SLA tables 2022-05-13 12:04:47 +02:00
Julian Brost
aeeab5918c Use default MySQL port if none is specified 2022-05-12 16:35:41 +02:00
Julian Brost
bfea92e039 Allow setting port for PostgreSQL Unix domain socket 
When you specify a socket path like /run/postgresql, the actual socket file is
located at /run/postgresql/.s.PGSQL.5432 where the number is the port number of
the server (as it's used for TCP). So it makes sense also allowing to set the
port for Unix domain sockets.

This commit also omits the port from the connection URI if none is given for
TCP connections, thus also using the default value in this case.
 2022-05-12 16:35:41 +02:00
Julian Brost
d5b55ee515 Retention: periodically log number of items removed 
Other components log this information periodically at level info, whereas the
retention feature was completely quiet at that level. It's logged at the
interval the retention cleanup is run rather than the global configured logging
interval since retention cleanup is done less often.
 2022-05-11 09:30:34 +02:00
Julian Brost
c1bbc037de
Merge pull request #471 from Icinga/embed.lua 
go:embed LUA script
 2022-05-10 12:53:07 +02:00
Julian Brost
078507122d
Merge pull request #459 from Icinga/424 
Ensure icingadb_instance#responsible=n
 2022-05-05 15:05:27 +02:00
Alexander A. Klimov
08b72cdf10 go:embed LUA script 
for the sake of syntax highlighting.
 2022-05-04 19:59:50 +02:00
Alexander A. Klimov
2f3a176bff On HA takeover set other instances' icingadb_instance#responsible = n 
for the case they crashed and didn't that by themselves.
 2022-05-04 19:41:03 +02:00
Alexander A. Klimov
67267a0f46 On HA handover: always set icingadb_instance#responsible=n 
even on lost heartbeat.
 2022-05-04 19:40:50 +02:00
Eric Lippmann
6cb452c2e6
Merge pull request #469 from Icinga/bugfix/retry-eagain 
Retry EAGAIN
 2022-05-04 16:38:06 +02:00
Julian Brost
ef0ccc3555 Retry EAGAIN 2022-05-03 14:30:31 +02:00
Alexander A. Klimov
4f9fda66d5 RDBMS: support *nix sockets as documented 2022-05-02 12:17:27 +02:00
Alexander A. Klimov
f843b06288 Do x.UnixMilli(), not utils.UnixMilli(x) 
and remove redundant utils.UnixMilli().
 2022-04-14 12:59:37 +02:00
Alexander A. Klimov
2bc92cbab7 UnixMilli(x): don't overflow on 4/11/2262 (UTC) 
but on 8/17/292278994 by not converting x into int64 ns and those to ms.
 2022-04-14 12:59:37 +02:00
Alexander A. Klimov
7b352c52db Merge Bulker and EntityBulker 2022-04-04 16:49:35 +02:00
Alexander A. Klimov
5b3a5cc163 Remove unused XMessageBulker 2022-03-30 13:40:26 +02:00
Alexander A. Klimov
b076f838bf Reuse generic com.Bulk() instead of redundant com.BulkXMessages() 2022-03-30 13:40:26 +02:00
Alexander A. Klimov
82cff812c6 Make Bulker generic 2022-03-30 13:40:26 +02:00
Eric Lippmann
c123276e2c Start history retention immediately 2022-03-22 15:55:04 +01:00
Eric Lippmann
17a7b026ce Add immediate option for periodic tasks 2022-03-22 15:55:04 +01:00
Ravi Kumar Kempapura Srinivasa
c4c89c2cae Allow history retention configuration 
Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2022-03-22 15:55:04 +01:00
Ravi Kumar Kempapura Srinivasa
5340805834 Introduce config.HistoryRetention 
Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2022-03-22 15:55:04 +01:00
Ravi Kumar Kempapura Srinivasa
5b99f74062 Introduce history.Retention 
Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2022-03-22 15:55:04 +01:00
Ravi Kumar Kempapura Srinivasa
5f4c36cc5b Introduce db.CleanupOlderThan() 
Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2022-03-21 16:48:12 +01:00
Eric Lippmann
3f6b303f4d Remove null bytes from strings 
PostgreSQL does not allow null bytes in varchar, char and
text fields.
 2022-03-10 17:07:06 +01:00
Alexander A. Klimov
d898681d3c Wrap "icingadb-*sql" in constants 
refs #136
 2022-03-10 17:07:06 +01:00
Alexander A. Klimov
b84fc62112 Simplify BulkChunkSplitPolicy 
refs #136
 2022-03-10 17:04:22 +01:00
Alexander A. Klimov
8da164c50f Throw BulkChunkSplitPolicy#Reset() away 
refs #136
 2022-03-10 17:04:22 +01:00
Alexander A. Klimov
10a70e8b71 Pass BulkChunkSplitPolicyFactory-s, not BulkChunkSplitPolicy-s 
refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
3eb14274dd DB#BuildDeleteStmt(): don't pre-rebind ?s 
refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
7f4b895ea9 NamedBulkExec(): allow custom BulkChunkSplitPolicy 
By the way avoid duplicate rows in the same upsert chunk to avoid Postgres
error 21000 (ON CONFLICT DO UPDATE command cannot affect row a second time).

refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
db2c3af769 BulkEntities(): allow custom BulkChunkSplitPolicy 
refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
d4f2c13d9c NewEntityBulker(): allow custom BulkChunkSplitPolicy 
refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
3854424a91 Introduce SplitOnDupId 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
6b59f2e47c Introduce NeverSplit 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
dccf02e11d Introduce BulkChunkSplitPolicy 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
5b87fd94ee Don't re-invent sqlx.DB#Rebind() 
refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
908bb42004 Postgres: upsert: only handle primary key conflicts 
refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
eca23a95ed DB#Build*Stmt(): quote table names 
Rationale: see https://dba.stackexchange.com/q/73136

refs #136
 2022-03-10 17:04:21 +01:00
Alexander A. Klimov
1c3bfcf99d Support Postgres 
refs #136
 2022-03-09 18:49:45 +01:00
Alexander A. Klimov
ad895b560d Handle Postgres-specific errors 
refs #136
 2022-03-09 18:49:45 +01:00
Alexander A. Klimov
aed0e9124f Avoid SQL syntax Postgres doesn't understand 
refs #136
 2022-03-09 18:49:45 +01:00
Alexander A. Klimov
879b683041 Add previous_soft_state to host_state and service_state 
refs #437
 2022-02-17 13:48:17 +01:00
Eric Lippmann
b65e427232
Merge pull request #432 from Icinga/bugfix/crash-after-interuption-of-mysql-connection-427 
HA#realize(): re-try transaction on temporary error
 2022-02-03 16:58:29 +01:00
Eric Lippmann
bf92290507 Don't embed TLS config 
When embedding TLS, the field name is implicitly specified as TLS,
which conflicts with the YAML key tls,
which currently does not allow TLS to be enabled.
I'm not entirely sure if this is a bug in goccy/go-yaml,
but if a struct field name other than TLS is specified,
the configuration works again.
 2022-02-03 13:10:16 +01:00
Eric Lippmann
59cf6ccc41 Rename TLS.Tls to TLS.Enable 
Enable is more self-documenting.
 2022-02-03 10:03:46 +01:00
Alexander A. Klimov
943133a4ac HA#realize(): re-try transaction on temporary error 
and don't just crash.

refs #427
 2022-02-02 13:23:37 +01:00
Noah Hilverling
12b99eda89 Make sure Redis connection pool is large enough 
The default Redis PoolSize is 10 * CPU, which isn't enough to run Icinga DB on machines with only 1 CPU.
Most of our open connections come from blocking XREADs. (e.g. Heartbeat, Runtime, Overdue, History)
 2021-12-09 15:55:24 +01:00
Eric Lippmann
68943a64c4
Merge pull request #410 from Icinga/bugfix/improve-notification-recipient 
Make NotificationRecipient struct readable
 2021-11-12 18:07:06 +01:00
Noah Hilverling
a938cfa175 Make NotificationRecipient struct readable 2021-11-12 16:33:33 +01:00
Eric Lippmann
f21f50e958 Reduce max_hmget_connections to 8 2021-11-12 16:29:59 +01:00
Eric Lippmann
69a0897847 Change default configuration file path 2021-11-12 16:29:59 +01:00
Julian Brost
f710c48394
Merge pull request #408 from Icinga/retry-broken 
Fix broken retry mechanics
 2021-11-12 14:32:31 +01:00
Eric Lippmann
a811c43727
Merge pull request #394 from Icinga/bugfix/unbulk-rtu 
RuntimeUpdates#Sync(): force FIFO for config updates
 2021-11-12 12:56:34 +01:00
Eric Lippmann
6a8163cdbc Runtime Updates: Use proper buffer channel sizes 2021-11-12 12:39:24 +01:00
Alexander A. Klimov
7d6474f6b5 Add special handling for entity bulks of size 1 
Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2021-11-12 12:39:22 +01:00
Alexander A. Klimov
d356909edc Add special handling for bulks of size 1 
Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2021-11-12 12:38:33 +01:00
Alexander A. Klimov
fecf332b8e RuntimeUpdates#Sync(): force FIFO for config updates 
so they don't interfer.

Co-authored-by: Eric Lippmann <eric.lippmann@icinga.com>
 2021-11-12 12:35:12 +01:00
Yonas Habteab
54c563d1c7 Add service_state.host_id column 2021-11-12 11:28:02 +01:00
Yonas Habteab
6faa84d94a Downtime: Add duration & scheduled_duration columns 2021-11-12 11:28:02 +01:00
Eric Lippmann
7f3621e980 Fix broken retry mechanics 
dc7511c introduced a parent context check to suppress logging if
it was canceled.
If it was not canceled, the check overwrites the variable err,
which resets every previous real error to nil,
which then leads to fatals because the OnError callback or
the IsRetryable function expect a non-nil error.
Since I cannot reproduce which logs should have been suppressed by
the changes, I removed them completely.
 2021-11-11 21:26:11 +01:00
Noah Hilverling
c19cfdf406 Schema: Prefix command_id with command type (check, event, notification) 
Signed-off-by: Eric Lippmann <eric.lippmann@icinga.com>
 2021-11-09 15:11:09 +01:00
Eric Lippmann
94abcef7bf
Merge pull request #388 from Icinga/bugfix/gpl2 
Replace Apache 2.0 licensed gopkg.in/yaml.v3 with MIT licensed github.com/goccy/go-yaml
 2021-11-09 11:13:29 +01:00
Eric Lippmann
ea74dc172a Rename periodic.Stoper to periodic.Stopper 2021-11-05 17:57:27 +01:00
Eric Lippmann
886c60c95a Use custom logger with 1 second interval for delta tests 2021-11-05 17:57:27 +01:00
Eric Lippmann
ccda48234e Use custom logger for accessing the interval for periodic logging 2021-11-05 17:57:22 +01:00
Eric Lippmann
8ec157e39b Add periodic logging for runtime updates 2021-11-05 17:52:48 +01:00
Eric Lippmann
6232773943 Use debug instead of info for some log messages 
These log messages are not relevant for the info level.
 2021-11-05 17:52:48 +01:00
Eric Lippmann
2d4b5419af Log which history sync started 2021-11-05 17:52:48 +01:00
Eric Lippmann
5fd4d35907 Remove syncing $subject log message 
This info message just pollutes the logs and
for debugging we log the execution anyway
 2021-11-05 17:52:46 +01:00
Eric Lippmann
43bcd2bbee Remove syncing $redisKey log message 
This info message just pollutes the logs and
for debugging we log the execution anyway.
 2021-11-05 17:52:11 +01:00
Eric Lippmann
8ce917d45a Remove waiting for heartbeat message 
If a heartbeat is pending,
we log it every 60 seconds anyway.
 2021-11-05 17:52:11 +01:00
Eric Lippmann
bd23f17eda Use pkg periodic for database logs 2021-11-05 17:52:10 +01:00
Eric Lippmann
8e7564b2aa Log which delta finished 2021-11-05 17:18:05 +01:00
Eric Lippmann
d6a28d7672 Use pkg periodic for history sync logs 2021-11-05 17:18:05 +01:00
Eric Lippmann
4b239d69bb Use debug instead of info for overdue refresh logs 2021-11-05 17:18:05 +01:00
Eric Lippmann
12525c7872 Use pkg periodic for overdue sync logs 2021-11-05 17:18:05 +01:00
Eric Lippmann
b10d038ba8 Use internal.LoggingInterval() 2021-11-05 17:18:05 +01:00
Eric Lippmann
dbb64a0de3 Log how many items to insert 2021-11-05 17:18:05 +01:00
Eric Lippmann
addfabbde1 Speak of items instead of rows 2021-11-05 17:18:05 +01:00
Eric Lippmann
5f1639aca2 Use pkg periodic for Redis logs 2021-11-05 17:18:05 +01:00
Eric Lippmann
a6e02e7f3c Introduce Counter.Total() 2021-11-05 17:18:05 +01:00
Eric Lippmann
c335a3c99c Introduce package periodic 2021-11-05 17:18:05 +01:00
Eric Lippmann
986e685ee0 Allow to configure interval for periodic logging 2021-11-05 17:18:05 +01:00
Eric Lippmann
b067ed2147 Introduce Counter.Reset() 2021-11-05 17:18:05 +01:00
Eric Lippmann
8a03745273 Speak of Icinga heartbeat not Icinga 2 heartbeat 2021-11-05 17:18:03 +01:00
Eric Lippmann
dc7511cd25 Don't log if context is canceled 2021-11-05 17:16:57 +01:00
Julian Brost
54dbe0cfbe
Merge pull request #391 from Icinga/bugfix/multi-environment 
Better handling of multiple environments
 2021-11-05 16:55:21 +01:00
Julian Brost
82cf600c55
Merge pull request #401 from Icinga/flawed-config-keys-and-validation 
Fix flawed config keys and validation
 2021-11-04 15:01:51 +01:00
Julian Brost
3342191b5e Exit on environment ID changes 
There's a small risk that when the environment ID changes, Icinga DB could
update write into the wrong environment in the database. Therefore,
https://github.com/Icinga/icinga2/pull/9036 introduced a new default
environment ID based on the CA public key so that there should be no cases
where it's required to change the actual environment ID. So if this happens
nonetheless, just bail out.
 2021-11-03 15:47:38 +01:00
Julian Brost
9b02b18f46 Use new environment ID 
https://github.com/Icinga/icinga2/pull/9036 introduced a new environment ID for
Icinga DB that's written to the icinga:stats stream as field
"icingadb_environment". This commit updates the code to make use of this ID
instead of the one derived from the Icinga 2 Environment constant.
 2021-11-03 15:47:38 +01:00
Eric Lippmann
a081927672 Only sync entities that belong to the current environment 
Previously, we selected each entity from the database.
Now we only select entities that belong to the current environment.
 2021-11-03 15:47:38 +01:00
Eric Lippmann
6b5aac9154 Config: Validate max_rows_per_transaction 2021-11-03 15:26:20 +01:00
Eric Lippmann
dac4a7246a Config: Validate max_placeholders_per_statement 2021-11-03 15:26:01 +01:00
Eric Lippmann
15eb9b471f Don't use CamelCase for config keys 2021-11-03 15:25:16 +01:00
Eric Lippmann
563aafaf90 Config: Validate xread_count 2021-11-03 15:23:40 +01:00
Eric Lippmann
084e0409bc Restart HA after environment change 
If the environment changes during runtime, we have to restart HA
in order to stop a possibly running config sync and start a new
one.
 2021-11-03 14:51:19 +01:00
Eric Lippmann
26a184d953 Make v1.Environment#Name types.String 
The default environment of Icinga is the empty string.
In Golang, the zero value of string is also the empty string.
But it makes sense to distinguish whether the name is not set
or set to the empty string. That is possible with this change.
 2021-11-03 14:49:56 +01:00
Eric Lippmann
54f9ef5a12 Insert environment 
With this change Icinga DB will insert the environment after each
heartbeat takeover if it does not already exist in the database as
the environment may have changed, although this is likely to happen
very rarely,

Instead of checking whether the environment already exists,
uses an INSERT statement that does nothing if it does.
 2021-11-03 14:49:56 +01:00
Eric Lippmann
4a659fd5c4 Add db.BuildIgnoreStmt() 2021-11-03 14:49:56 +01:00
Julian Brost
f290a0c9b7
Merge pull request #392 from Icinga/feature/history-deterministic-ids 
Make {NotificationHistory,StateHistory,History*}#Id UUID -> SHA1
 2021-11-03 13:18:11 +01:00
Alexander A. Klimov
b39eac660f Replace Apache 2.0 licensed gopkg.in/yaml.v3 with MIT licensed github.com/goccy/go-yaml 
... not to have GPLv2<->Apache 2.0 (app<->deps) license conflicts.
 2021-11-03 12:22:21 +01:00
Alexander A. Klimov
d903d05c82 Make History*#Id UUID -> SHA1 2021-11-03 12:15:25 +01:00
Alexander A. Klimov
52ae34a6f8 Make {NotificationHistory,StateHistory}#Id UUID -> SHA1 2021-11-03 12:15:25 +01:00
Eric Lippmann
e433aa7ec3 Move custom var sync to a new method 2021-10-26 09:31:41 +02:00
Eric Lippmann
65440ee8fe Expect no custom var clear events during runtime 2021-10-26 09:31:41 +02:00
Eric Lippmann
b48792cf36 Handle flat custom vars explicitly in runtime updates 
This also requires explicit handling of custom variables as we need
to multiplex the original values to handle flat custom variables.
 2021-10-26 09:31:41 +02:00
Eric Lippmann
4d65c62f77 Handle contracts.Initer in common.NewSyncSubject() 
contracts.EntitiyFactoryFunc.WithInit() checked for
contracts.Initer every time.
Now it is only done once in common.NewSyncSubject().
 2021-10-26 09:31:36 +02:00
Eric Lippmann
c78326ad1b Use SyncSubject in RuntimeUpdates.Sync() 2021-10-26 09:27:00 +02:00
Eric Lippmann
fe6915447e Add method SyncSubject.Name() 2021-10-26 09:27:00 +02:00
Eric Lippmann
a73a882c6f Introduce function ExpandCustomvars() 2021-10-26 09:27:00 +02:00
Eric Lippmann
d017a05d05 Export DB.getSemaphoreForTable() 2021-10-26 09:27:00 +02:00
Eric Lippmann
16dd4663ad Move method DB.getSemaphoreForTable() 2021-10-26 09:27:00 +02:00
Eric Lippmann
44b45fc429 Use context from errgroup 2021-10-26 09:27:00 +02:00
Alexander A. Klimov
b5e024e68d Sync state runtime updates ASAP 
I.e. don't wait for the complete initial sync first.
 2021-10-14 10:15:28 +02:00
Eric Lippmann
537a4cf37f Allow to configure the logging output 2021-10-13 09:46:12 +02:00
Eric Lippmann
a9afcea25c Allow systemd-journald (and console) as log outputs 2021-10-13 09:46:12 +02:00
Eric Lippmann
b582995e37 Introduce zapcore.Core that sends logs to systemd-journald 2021-10-13 09:46:12 +02:00
Eric Lippmann
9e49b62c4d Use the app name as the default logger name 2021-10-13 09:46:12 +02:00
Eric Lippmann
3061e3d0c5 Introduce function utils.AppName() 2021-10-13 09:46:12 +02:00
Eric Lippmann
66d34b4a9f Add ConvertCamelCase utility function 2021-10-13 09:46:12 +02:00
Eric Lippmann
bdeed69337 Move logging from internal to pkg 2021-10-13 09:20:55 +02:00
Eric Lippmann
d8ba0c374a
Merge pull request #364 from Icinga/feature/history-sync-foreign-keys 
Add foreign key constraints to history tables
 2021-10-07 18:38:33 +02:00
Julian Brost
7c782e3eb8 History sync: use information from notification stream for user_notification_history table 2021-10-05 18:35:02 +02:00
Julian Brost
8b4e4d68a6 History sync: use indefinitely blocking XREAD 
Just like we do it throughout the rest of the code.
 2021-10-05 18:35:02 +02:00
Julian Brost
bfcc324535 History sync: rewrite to use a sequential pipeline 
This is in preparation for adding foreign key constraints to the history
tables. For this, is is required to insert the rows into the different history
tables in a defined order.
 2021-10-05 18:35:02 +02:00
Julian Brost
82530c771d Redis/DB: export options member 
This change allows the history sync to use values configured in these options.
 2021-10-05 18:34:55 +02:00
Julian Brost
c5af0cd287 HA: only set realize timeout when active 
When inactive, this is the only query running so it has to retry for longer to
eventually trigger a fatal error if the database is gone for too long (5
minutes at the moment).
 2021-10-04 16:58:35 +02:00
Julian Brost
239d2ea410 HA: after heartbeat expiry, stop writing to database and hand over 
If it's not possible for Icinga DB to write through the heartbeat within its
validity period it cannot signal to other instances that it still is alive and
has the hand over. There's also no point in retrying for this individual
heartbeat any longer.
 2021-10-04 16:58:35 +02:00
Julian Brost
a34aef4fc5 retry: if stopped due to outer context, return that error 
If there is an outer context that is canceled or exceeds its deadline before
the internal timeout is reached, its error should be passed on as the failure
didn't happen due to retry giving up.
 2021-10-04 16:58:35 +02:00
Julian Brost
217ab03e59 heartbeat: wrap messages with a timestamp 
Track when a heartbeat was received to allow other components to check when it
will expire.
 2021-10-04 16:58:35 +02:00
Julian Brost
8b2cb3acb8 heartbeat: use a single channel for all beat/loss events 
Using Cond does not allow to reliably catch all events as one will only receive
events that occour after starting to listen. For heartbeat loss events it's
import to reliably catch them to not remain in an HA active state incorrectly.

fixes #360
 2021-10-04 16:36:09 +02:00
Julian Brost
a1b78e0f23 Add XMessageBulker 
Generics would be nice but we don't have them yet unfortunatly, so for now, yet
another copy of Bulker (as it already exists for EntityBulker).
 2021-10-04 14:44:50 +02:00
Alexander Aleksandrovič Klimov
d99e0586a5
Merge pull request #236 from Icinga/feature/tls 
Support TLS
 2021-10-04 12:27:21 +02:00
Alexander A. Klimov
82c26b187e Support TLS 2021-09-30 12:25:23 +02:00
Ravi Kumar Kempapura Srinivasa
414057830e Allow to configure logging in the YAML configuration 2021-09-28 17:30:11 +02:00
Ravi Kumar Kempapura Srinivasa
acde6ade69 Introduce Logging config struct 2021-09-28 17:30:11 +02:00
Julian Brost
65074f5755
Merge pull request #370 from Icinga/feature/icingadb-scheduling_source-160 
Include CheckResult#scheduling_source in state and history
 2021-09-27 17:32:42 +02:00
Julian Brost
6e3df7d63b
Merge pull request #373 from Icinga/feature/single-threaded-delta 
Rewrite delta to use only a single goroutine
 2021-09-24 16:53:23 +02:00
Julian Brost
0c9fb2f22f
Merge pull request #369 from Icinga/feature/log-reconnects-351 
Log all different failed and recovered reconnects to backends
 2021-09-24 11:58:34 +02:00
Julian Brost
66d9b0e6e6 Rewrite delta to use only a single goroutine 2021-09-24 11:52:15 +02:00
Julian Brost
1e9a88bee6 Add tests and benchmarks for delta computation 2021-09-24 11:52:13 +02:00
Alexander A. Klimov
b4bfee92d9 Log all recovered reconnects to backends 
... to give the admin the all-clear.

refs #351
 2021-09-23 16:07:41 +02:00
Julian Brost
e0c903bfdc Redis HYield: remove duplicates returned by HSCAN 
fixes #349
 2021-09-23 14:36:51 +02:00
Julian Brost
4457f9f440
Merge pull request #365 from Icinga/data-races 
Fix data races
 2021-09-23 12:32:19 +02:00
Eric Lippmann
454381c820 Use uint64 instead of Counter 
Use uint64 as there is no longer any concurrent access.
 2021-09-23 12:18:08 +02:00
Eric Lippmann
98202e1257 Use buffered channel 
Use a buffered channel so that the next HSCAN call does not have
to wait until the previous result has been processed.
 2021-09-23 09:37:31 +02:00
Eric Lippmann
c1e722f5fa Do not close channel too early 
This fixes a data race where the pairs channel was closed too early
when the context is canceled and therefore the outer errgroup
returns from Redis operations before Wait() is called on the inner
errgroup. Unfinished Go methods in the inner errgroup would then
try to work on a closed channel.
 2021-09-23 09:37:31 +02:00
Eric Lippmann
7351559793 Use pointer receiver for Counter.Val() 
This fixes a data race as Val() was previously operating on a copy
of the counter while Inc() and Add() may haved changed the original
value.
 2021-09-23 09:37:31 +02:00
Eric Lippmann
9ce2cff5c0 Introduce WaiterFunc type 
The WaiterFunc type is an adapter to allow the use of ordinary
functions as Waiter.
 2021-09-23 09:37:31 +02:00
Julian Brost
17321cdfc3 Fix use of wrong log function on heartbeat loss 
Has to use the Warnw function as it passes additional zap attributes.
 2021-09-23 09:27:26 +02:00
Alexander A. Klimov
82d8f830af Include CheckResult#scheduling_source in state and history 
refs #160
 2021-09-22 17:30:13 +02:00
Alexander Aleksandrovič Klimov
585d1e6bb5
Merge pull request #368 from Icinga/bugfix/icinga-db-does-not-exit-when-reconnecting-to-the-database-350 
On shutdown: give up HA handover after 3s, not 5m
 2021-09-22 16:22:51 +02:00
Alexander A. Klimov
f554fa9dfe Log all different failed reconnects to backends 
E.g. the first "connection refused" and the first "hostname mismatch".

refs #351
 2021-09-22 16:15:37 +02:00
Alexander A. Klimov
321db0eecf Introduce Settings#OnSuccess 
refs #351
 2021-09-22 15:35:08 +02:00
Alexander A. Klimov
653df82a1e Introduce Settings#OnError 
refs #351
 2021-09-22 15:34:39 +02:00
Alexander A. Klimov
ea7668d99a HA#Close(): allow custom context 
refs #350
 2021-09-22 14:12:27 +02:00
Alexander A. Klimov
5a146645f2 HA#removeInstance(): allow custom context 
refs #350
 2021-09-22 14:12:27 +02:00
Alexander A. Klimov
8d57ec107a WithBackoff(): aggregate optional settings in one struct 
refs #351
 2021-09-22 13:37:12 +02:00
Alexander Aleksandrovič Klimov
4371d04d5e
Merge pull request #356 from Icinga/bugfix/mustpackany 
Clarify what MustPackAny() does
 2021-09-21 16:16:56 +02:00
Alexander Aleksandrovič Klimov
956590cb99
Merge pull request #301 from Icinga/feature/scheduled_by 
Introduce downtime#scheduled_by
 2021-09-02 10:31:59 +02:00
Julian Brost
be9054628a Ensure extra config options are properly initialized 
YAML is decoded by the structure of the YAML source document, not the Go
destination data structure. Therefore, the old code did not always call
UnmarshalYAML() on all sub-structs. Therefore, defaults were not always set but
zero values were used, resulting in all kind of strange behavior.

This commit changes the code so that it no longer relies on individual
UnmarshalYAML() functions to set the defaults for each sub-struct but instead
just sets all of them when creating the surrounding Config instance. It also
moves the config validation to separate Validate() functions.
 2021-09-01 18:49:38 +02:00
Alexander A. Klimov
3f8703310c Clarify what MustPackAny() does 
It always packs a slice, even if only one item given.
 2021-08-20 11:54:00 +02:00
Eric Lippmann
0bf4f4df0a
Merge pull request #343 from Icinga/config-options 
Separate required and optional configuration for database and Redis
 2021-08-10 12:33:22 +02:00
Alexander Aleksandrovič Klimov
5a91d8acf6
Merge pull request #338 from Icinga/feature/support-icingadb-version-again-335 
Support --version
 2021-08-10 11:17:51 +02:00
Eric Lippmann
fbbb9bfacd Don't allow 0 for timeout redis option 
0 stands for deactivate, which makes no sense here.
 2021-08-10 09:29:27 +02:00
Eric Lippmann
8232000524 Don't allow 0 for max_connections database option 
0 stands for deactivate, which makes no sense here.
 2021-08-10 08:55:24 +02:00
Eric Lippmann
50473bca70 Remove UnmarshalYAML 
Config options are no longer inlined.
 2021-08-09 22:06:55 +02:00
Eric Lippmann
1c386c9c2f Don't inline Database options 
There is now the options key to separate required and optional
configuration. Before, both were mixed.
 2021-08-09 22:06:50 +02:00
Eric Lippmann
8927e942f1 Remove UnmarshalYAML 
Config options are no longer inlined.
 2021-08-09 22:06:50 +02:00
Eric Lippmann
559b27cd8b Don't inline Redis options 
There is now the options key to separate required and optional
configuration. Before, both were mixed.
 2021-08-09 21:48:27 +02:00
Eric Lippmann
f9e12d9df7 Move method 2021-08-09 21:45:08 +02:00
Alexander A. Klimov
825dcbc817 Introduce downtime#scheduled_by 2021-08-09 20:12:10 +02:00
Eric Lippmann
783b3a6bfe
Merge pull request #323 from Icinga/feature/downtime-parent-downtime-id 
Downtime: Add parent_id
 2021-08-09 13:43:16 +02:00
Eric Lippmann
0a521a6e4a Add missing doc in meta 2021-08-09 10:30:53 +02:00
Eric Lippmann
1d5ae198aa Add missing doc in meta 2021-08-09 10:30:53 +02:00