upstream/icingadb
1
0
Fork 0
mirror of https://github.com/Icinga/icingadb.git synced 2026-05-28 04:35:54 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
1971 commits 29 branches 11 tags 14 MiB
Commit graph

602 commits

Author SHA1 Message Date
Eric Lippmann
7fa8147303 Merge pull request #34 from lippserd/bugfix/cv-flat-nil 
Flat CVs: represent null as "null", not "<nil>"
 2021-05-12 14:16:49 +02:00
Alexander A. Klimov
188ce4cb4b Flat CVs: represent null as "null", not "<nil>" 2021-05-12 12:08:13 +02:00
Julian Brost
f82de0c1a9 Make Redis/MySQL concurrency and batch sizes configurable 2021-05-12 11:54:36 +02:00
Eric Lippmann
58f80bc8d8 Introduce objectpacker.MustPackAny() 2021-05-11 16:40:08 +02:00
Eric Lippmann
ac9aa0365a Remove time taken debug log for named queries 2021-05-11 16:40:08 +02:00
Eric Lippmann
bf27824980 Remove NewCustomvar from Factories 
We now explicitly sync it in main.go.
 2021-05-11 16:40:08 +02:00
Eric Lippmann
e71833defb Flatten: Fix keys 2021-05-11 16:40:08 +02:00
Eric Lippmann
52e5eb774e Sync customvar flat 2021-05-11 16:40:08 +02:00
Eric Lippmann
f4ec939b9f Add func icingadb.v1.FlattenCustomvars() 2021-05-11 16:40:08 +02:00
Eric Lippmann
766740974f Precise log messages 2021-05-11 16:40:08 +02:00
Eric Lippmann
850de4d371 Remove log message 2021-05-11 16:40:08 +02:00
Eric Lippmann
eea1d0e486 Fix race 2021-05-11 16:40:08 +02:00
Eric Lippmann
9c86198302 Remove icingadb.Sync.fromRedis() 2021-05-11 16:40:08 +02:00
Eric Lippmann
5550c47be3 Add NewCustomvarFlat() 2021-05-11 16:40:08 +02:00
Eric Lippmann
7c70a72991 Introduce icingadb.Sync.ApplyDelta() 2021-05-11 16:40:08 +02:00
Eric Lippmann
34363e19ac Expect common.SyncSubject in icingadb.Delta 2021-05-11 16:40:08 +02:00
Eric Lippmann
442e04bbf1 Use type common.SyncSubject 2021-05-11 16:40:08 +02:00
Eric Lippmann
51d2532f18 Introduce func icingaredis.Client.YieldAll() 2021-05-11 16:40:08 +02:00
Eric Lippmann
961a3650e8 Introduce type common.SyncSubject 2021-05-11 16:40:08 +02:00
Eric Lippmann
420193f228 Merge pull request #50 from lippserd/bugfix/obj-pack-bytes 
PackAny(): support types.Binary
 2021-05-11 13:53:54 +02:00
Alexander A. Klimov
40c18e3492 PackAny(): Golangify spec doc 2021-05-10 14:01:52 +02:00
Alexander A. Klimov
a6da8ab90a PackAny(): pack [I]byte as string in map keys as well 2021-05-10 13:15:34 +02:00
Alexander A. Klimov
c8fa629f32 PackAny(): restore unit test case 2021-05-10 12:38:54 +02:00
Alexander A. Klimov
1b4b207465 PackAny(): enhance code docs 2021-05-10 12:35:23 +02:00
Alexander A. Klimov
9df2f9d7e6 PackAny(): panic() explicitly, not due to a programming error 2021-05-10 12:13:24 +02:00
Eric Lippmann
a1ceeeb9eb Merge pull request #36 from lippserd/feature/runtime-updates 
Implement runtime updates
 2021-05-07 14:37:56 +02:00
Alexander A. Klimov
23127e3245 PackAny(): disallow types recursively more strictly 2021-05-07 12:38:18 +02:00
Alexander A. Klimov
a66cfca9c8 PackAny(): support numbers only as float64 2021-05-07 12:16:09 +02:00
Noah Hilverling
91a8228e2d Implement runtime updates 2021-05-07 12:04:41 +02:00
Noah Hilverling
cf195c9b98 Fix UsergroupMember.UserId JSON tag 2021-05-07 09:18:41 +02:00
Alexander A. Klimov
ab43673593 PackAny(): support types.Binary 2021-05-06 18:00:47 +02:00
Alexander A. Klimov
bb850ddb5d PackAny(): pack []byte as string, not array of numbers 2021-05-06 17:53:54 +02:00
Noah Hilverling
ca9936e5b6 icingadb.DB: Rename functions that take a channel to *Streamed() 2021-05-04 14:30:53 +02:00
Noah Hilverling
25558f9827 icingadb.DB: Add Delete() and DeleteStreamed() 2021-05-04 14:30:23 +02:00
Noah Hilverling
4b56dd1de2 com.Bulker: Change type from contracts.Entity to interface{} 2021-05-04 11:46:33 +02:00
Noah Hilverling
507a414ca7 Add com.EntityBulker 2021-05-04 11:46:33 +02:00
Noah Hilverling
b05a00a8d5 icingaredis.Client: Add StreamLastId() 2021-05-04 11:46:33 +02:00
Noah Hilverling
26c889ad66 types.NotificationTypes: Implement UnmarshalText() 2021-05-04 11:46:33 +02:00
Noah Hilverling
9e3a8b4d1d types.NotificationStates: Implement UnmarshalText() 2021-05-04 11:46:33 +02:00
Noah Hilverling
564d3d594e types.Int: Implement UnmarshalText() 2021-05-04 11:46:33 +02:00
Noah Hilverling
96835ace04 Structify: Allow for string ptr (e.g. NameCi) 2021-05-04 11:46:33 +02:00
Noah Hilverling
094e374070 NameCiMeta: Add missing json tag to NameMeta 2021-05-04 11:46:33 +02:00
Alexander A. Klimov
429ea1ca48 Introduce PackAny() 2021-05-04 11:09:24 +02:00
Noah Hilverling
7bc8bddc5a Add service state 2021-05-04 09:55:43 +02:00
Noah Hilverling
4dc38c42e7 Add host state 2021-05-04 09:55:43 +02:00
Noah Hilverling
ebf6698540 Add v1.State 2021-05-04 09:55:43 +02:00
Noah Hilverling
0dac6aec10 Add types.AcknowledgementState 2021-05-04 09:25:06 +02:00
Noah Hilverling
39589dcb7d types.StateType: Implement UnmarshalJson() 2021-05-04 09:25:06 +02:00
Noah Hilverling
a0a1eec35f Change redis key prefix to 'icinga:*' instead of 'icinga:config:*' 2021-05-04 09:22:38 +02:00
Julian Brost
a7e5395f66 Read icinga:dump stream and respect dump signals 2021-05-03 11:19:26 +02:00
Julian Brost
ab24bdd1f7 Main loop: return if ctx has an error 
If ctx.Done(), there's no point in looping again.
 2021-04-28 16:05:56 +02:00
Julian Brost
d138b8e0ac Handle SIGINT/SIGTERM/SIGHUP 
On any of these signals, HA removes the row of the local instance from
icingadb_instance. This allows another instance to take over immediately.
 2021-04-28 12:01:02 +02:00
Eric Lippmann
3a4c96fa2e Introduce v1.Factories 2021-04-27 23:34:35 +02:00
Eric Lippmann
adcf9a6cfd Add EntityFactoryFunc.WithInit() 2021-04-27 23:33:49 +02:00
Alexander A. Klimov
0ec2b6ca60 Return ctx.Err() on ctx.Done() 2021-04-26 14:15:10 +02:00
Eric Lippmann
cab03bd8f6 Merge pull request #19 from lippserd/bugfix/hmyield 
HMYield(): handle missing values
 2021-04-22 10:20:21 +02:00
Alexander A. Klimov
3304f0486c HMYield(): handle missing values 2021-04-21 15:26:31 +02:00
Alexander A. Klimov
b8d2372899 Sync#Sync(): don't run no-ops 2021-04-19 18:23:18 +02:00
Alexander A. Klimov
4dc935534c Fix missing import 2021-04-19 16:48:02 +02:00
Eric Lippmann
0c8a9139b5 Merge pull request #28 from lippserd/bugfix/delta-race 
Delta#start(): avoid a race across maps by using a mutex
 2021-04-19 16:20:42 +02:00
Eric Lippmann
3ea98313c3 Merge pull request #17 from lippserd/feature/history-sync 
Sync history
 2021-04-15 22:23:49 +02:00
Alexander A. Klimov
bf1a77a67c Drop utils.SyncMap*() 2021-04-12 17:41:18 +02:00
Alexander A. Klimov
fac47fb330 Delta: don't over-lock 2021-04-12 17:41:18 +02:00
Alexander A. Klimov
3974a7bd4f Introduce EntitiesById 2021-04-12 17:29:32 +02:00
Alexander A. Klimov
7ccf627df6 Sync history 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
9aa1070db0 Introduce contracts.TableNamer 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
3fbc9fa25f DB#Upsert(): workaround jmoiron/sqlx#694 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
4e87ca6de3 DB#Upsert(): allow to update not all columns 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
ebfabaffc2 Introduce DB#Upsert() 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
500dbac66a Deduplicate DB#Create() and DB#Update() 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
a9a4450068 DB#NamedBulkExec*(): reduce channel requirements 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
df0124de09 DB#NamedBulkExec(): optionally inform about successfully inserted rows 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
b13d2c3cd7 Restrict Bulker to contracts.Entity 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
581cac27aa Make types.UnixMilli an encoding.TextUnmarshaler 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
270dfa6159 Make types.String an encoding.TextUnmarshaler 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
bfd22b1f39 Make types.Bool an encoding.TextUnmarshaler 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
fb4fd4c964 Introduce types.StateType 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
ba69cef6cb Introduce types.NotificationType 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
c86a8df20b Introduce types.Float 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
f2adf2e8c1 Introduce types.UUID 2021-04-12 13:28:03 +02:00
Alexander A. Klimov
ee023b1202 Introduce utils.MakeMapStructifier() 2021-04-12 13:28:03 +02:00
Eric Lippmann
b36d198075 Merge pull request #29 from lippserd/bugfix/address-bin 
Host: sync also address{,6}_bin
 2021-04-12 10:53:49 +02:00
Eric Lippmann
d96b5938b1 Merge pull request #27 from lippserd/feature/json-name_ci 
NameCiMeta#NameCi: don't (unneccessarily) decode from JSON
 2021-04-12 10:52:36 +02:00
Alexander A. Klimov
5f4d04e5a7 Host: sync also address{,6}_bin 2021-04-08 17:33:43 +02:00
Alexander A. Klimov
d36928afd3 Delta#start(): avoid a race across maps by using a mutex 
Imagine an Icinga restart w/o any config changes and a full dump already being done.
One goroutine reads Redis, the other the database.
Both get the same object at the same time and check it in the map of the other goroutine - not present.
So they store it in their own map.
I.e. the same object hasn't been changed, but has to be deleted and inserted.
If the insert comes first, that causes a duplicate key error.
 2021-04-08 16:28:42 +02:00
Alexander A. Klimov
0b660a7e7a NameCiMeta#NameCi: don't (unneccessarily) decode from JSON 2021-04-08 13:50:38 +02:00
Alexander A. Klimov
e092e6de68 shouldRetry(): retry on driver.ErrBadConn as well 2021-04-07 13:56:43 +02:00
Alexander A. Klimov
f5003e6119 Actually use re-trying SQL driver 2021-04-07 13:55:40 +02:00
Eric Lippmann
d1b1693d14 Merge pull request #23 from lippserd/bugfix/comment-entrytype 
Introduce types.CommentType
 2021-04-06 10:10:28 +02:00
Eric Lippmann
cf110d74bf Merge pull request #18 from lippserd/bugfix/notification-bitmask 
Correctly parse notification bitmask types
 2021-04-06 10:07:54 +02:00
Eric Lippmann
8345202fc6 Merge pull request #21 from lippserd/bugfix/json-inline 
EntityWithChecksum: fix missing `json:",inline"`
 2021-03-30 10:57:11 +02:00
Eric Lippmann
32ece5ea43 Merge pull request #20 from lippserd/bugfix/cmd-arg-bool-null 
Make CommandArgument an Initer
 2021-03-30 10:56:28 +02:00
Eric Lippmann
63d8bf8e24 Merge pull request #15 from lippserd/feature/timeperiod 
Sync timeperiods as well
 2021-03-30 10:12:33 +02:00
Alexander A. Klimov
4743693872 Introduce types.CommentType 2021-03-26 14:16:20 +01:00
Alexander A. Klimov
4a90674823 EntityWithChecksum: fix missing json:",inline" 2021-03-23 19:26:49 +01:00
Alexander A. Klimov
96a93847f4 Make CommandArgument an Initer 
... to auto-set the required defaults.
 2021-03-23 17:23:10 +01:00
Alexander A. Klimov
d782d41b79 Introduce types.NotificationStates 2021-03-23 16:27:29 +01:00
Alexander A. Klimov
c2cdb82fc4 Introduce types.NotificationTypes 2021-03-23 16:27:29 +01:00
Eric Lippmann
9525ef476e Merge pull request #13 from lippserd/feature/notification 
Sync notifications as well
 2021-03-23 10:09:06 +01:00
Eric Lippmann
b9673e1a94 Merge pull request #11 from lippserd/feature/comment 
Sync comments as well
 2021-03-23 10:08:01 +01:00
Alexander A. Klimov
fd2e068de6 Sync timeperiods as well 2021-03-22 14:07:52 +01:00
Alexander A. Klimov
7ab45082ad Sync notifications as well 2021-03-22 14:06:51 +01:00
Alexander A. Klimov
3ec88ac53d Sync comments as well 2021-03-22 14:04:58 +01:00
Alexander A. Klimov
b90bc86bb4 Sync zones as well 2021-03-22 14:03:46 +01:00
Eric Lippmann
49d50a779e Merge pull request #12 from lippserd/feature/downtime 
Sync downtimes as well
 2021-03-22 13:37:35 +01:00
Eric Lippmann
dd1fad4b13 Merge pull request #10 from lippserd/feature/command 
Sync commands as well
 2021-03-22 13:36:46 +01:00
Eric Lippmann
9f936eba95 Merge pull request #7 from lippserd/feature/groups 
Sync groups as well
 2021-03-22 13:36:12 +01:00
Alexander A. Klimov
13c672ce6e Sync downtimes as well 2021-03-18 13:56:07 +01:00
Alexander A. Klimov
a56d312c81 Sync commands as well 2021-03-18 13:51:33 +01:00
Alexander A. Klimov
248e5ed0a8 Introduce Int 2021-03-18 13:50:55 +01:00
Alexander A. Klimov
e91f2a7aa5 Introduce String 2021-03-18 13:50:55 +01:00
Eric Lippmann
e4539d90fc Merge pull request #8 from lippserd/feature/endpoint 
Sync endpoints as well
 2021-03-18 13:42:20 +01:00
Alexander A. Klimov
71d1b4a9f4 Sync endpoints as well 2021-03-18 12:32:09 +01:00
Alexander A. Klimov
055e63e272 Sync groups as well 2021-03-18 12:31:11 +01:00
Alexander A. Klimov
0dd5e54b14 Sync users as well 2021-03-18 12:19:23 +01:00
Alexander A. Klimov
3a66f0372c Introduce CustomvarMeta 2021-03-18 12:19:20 +01:00
Alexander A. Klimov
66d155cdb3 Group related types 2021-03-18 12:15:12 +01:00
Eric Lippmann
c2fe0dfe7c Merge pull request #16 from lippserd/feature/channels 
Make channels more specific
 2021-03-17 10:03:24 +01:00
Eric Lippmann
f9648b00c5 Merge pull request #14 from lippserd/feature/url 
Sync URLs as well
 2021-03-17 10:02:20 +01:00
Eric Lippmann
c5f1b88c0c Merge pull request #4 from lippserd/feature/initer 
Make NameCiMeta an Initer
 2021-03-17 09:56:10 +01:00
Eric Lippmann
dfd904b064 Merge pull request #1 from lippserd/ha 
HA improvements
 2021-03-16 10:12:23 +01:00
Alexander A. Klimov
4dffbad76e Make channels more specific 2021-03-15 16:34:58 +01:00
Alexander A. Klimov
aa3dfc2afc Sync URLs as well 2021-03-12 16:44:19 +01:00
Alexander A. Klimov
02b15e01a3 Host: re-use Checkable 2021-03-12 10:59:40 +01:00
Alexander A. Klimov
cf2fdfe381 New*(): don't re-do Init() 2021-03-12 10:55:28 +01:00
Alexander A. Klimov
ecf3dd74ae Make NameCiMeta an Initer 2021-03-12 10:55:09 +01:00
Julian Brost
6ccbc7d091 HA: only execute query to remove old instances once 2021-03-08 13:01:52 +01:00
Julian Brost
a474c83651 Replace persisted instance ID with a random one and remove old rows from icingadb_instance 2021-03-08 13:01:52 +01:00
Julian Brost
ab268bfbb6 Always write HA heartbeat 2021-03-08 13:01:52 +01:00
Julian Brost
4293a51c88 Only signal HA takeover if a takeover was attempted 2021-03-08 13:01:52 +01:00
Julian Brost
476d2bc20e Insert endpoint_id into icingadb_instance 2021-03-08 13:01:52 +01:00
Eric Lippmann
41cfa9e0df Add first set of types to sync 2021-03-04 00:49:23 +01:00
Eric Lippmann
34603b5e1d Introduce meta types 2021-03-04 00:49:23 +01:00
Eric Lippmann
bb9a2b0251 Implement sync 2021-03-04 00:49:23 +01:00
Eric Lippmann
262749c575 Add type icingadb.HA 2021-03-04 00:49:23 +01:00
Eric Lippmann
77267fa60c Introducte type icingaredis.Heartbeat 2021-03-04 00:49:23 +01:00
Eric Lippmann
50b3b6ea30 Add types necessary for heartbeat and HA 2021-03-04 00:49:23 +01:00
Eric Lippmann
5f12ca9b82 Add type icingaredis.Client 
icingaredisClient is a wrapper around redis.Client with streaming
and logging capabilities.
 2021-03-03 21:04:50 +01:00
Eric Lippmann
facfc16958 Introduce pkg com 
pkg com provides concurrency and channel related utilities.
 2021-03-03 15:59:52 +01:00
Eric Lippmann
a696157f47 Introduce flatten 
Flatten creates flat, one-dimensional maps from arbitrarily nested
values, e.g. JSON.
 2021-03-03 15:55:54 +01:00
Eric Lippmann
4131290641 Auto-reconnect database conns 2021-03-03 15:30:51 +01:00
Eric Lippmann
61f8e7f915 Introduce retry functionality 
With exponential backoff and context support.
 2021-03-03 15:27:20 +01:00
Eric Lippmann
a35eaefae9 Increase timeout to 60s for database connections 2021-03-03 15:20:27 +01:00
Eric Lippmann
6cd2c20e22 Add type icingadb.DB 
DB is a wrapper around sqlx.DB with bulk execution,
statement building, streaming and logging capabilities.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
af89b2652b Add config utils 
This commit introduces utility functions for parsing and validating
CLI flags and YAML configuration files.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
140f46af0e Add type config.Redis 
Redis defines Redis client configuration.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
03f26b5b18 Add type config.Database 
Database defines database client configuration.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
53f51a053d Add type types.Bool 
Bool represents a bool for ENUM ('y', 'n'), which can be NULL.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
eecc9bf528 Add type types.UnixMilli 
UnixMilli is a nullable millisecond UNIX timestamp in databases and
JSON. We work with milliseconds throughout the Icinga DB, so this
should be the type for every timestamp.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
cbdde4b15f Add type types.Binary 
Binary is a nullable byte string used for IDs and checksums or
keys in general in databases and Redis. For JSON, Binary is
represented as hex.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
03e6e9d0f3 Add utils 
This commit introduces various utility functions.
 2021-03-03 15:20:27 +01:00
Eric Lippmann
5f5028e637 Add contracts 
This commit introduces the abstract type Entity and related components.
Entity has to be implemented by all types that Icinga DB should
synchronize.
 2021-03-03 15:19:55 +01:00