prometheus/storage/remote
Bartlomiej Plotka a73202012b
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Compliance testing (push) Waiting to run
CI / Build Prometheus for common architectures (push) Waiting to run
CI / Build Prometheus for all architectures (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
tsdb/wlog[PERF]: optimize WAL watcher reads (up to 540x less B/op; 13000x less allocs/op) (#18250)
See the detailed analysis https://docs.google.com/document/d/1efVAMcEw7-R_KatHHcobcFBlNsre-DoThVHI8AO2SDQ/edit?tab=t.0

I ran extensive benchmarks using synthetic data as well as real WAL segments pulled from the prombench runs.

All benchmarks are here https://github.com/prometheus/prometheus/compare/bwplotka/wal-reuse?expand=1

* optimization(tsdb/wlog): reuse Ref* buffers across WAL watchers' reads

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimization(tsdb/wlog): avoid expensive error wraps

Signed-off-by: bwplotka <bwplotka@gmail.com>

* optimization(tsdb/wlog): reuse array for filtering

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fmt

Signed-off-by: bwplotka <bwplotka@gmail.com>

* lint fix

Signed-off-by: bwplotka <bwplotka@gmail.com>

* tsdb/record: add test for clear() on histograms

Signed-off-by: bwplotka <bwplotka@gmail.com>

* updated WriteTo with what's currently expected

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
2026-03-11 09:17:13 +00:00
..
azuread chore: fix emptyStringTest issues from gocritic (#18226) 2026-03-04 08:24:50 +01:00
googleiam Replace deprecated google.CredentialsFromJSON with option.WithAuthCredentialsFile 2026-02-10 13:03:14 -08:00
otlptranslator/prometheusremotewrite chore: fix emptyStringTest issues from gocritic (#18226) 2026-03-04 08:24:50 +01:00
chunked.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
chunked_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
client.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
client_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
codec.go feat(tsdb): new AppenderV2 and AtST interface for chunks 2026-01-14 13:15:09 +01:00
codec_test.go feat(tsdb): new AppenderV2 and AtST interface for chunks 2026-01-14 13:15:09 +01:00
dial_context.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
dial_context_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
ewma.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
intern.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
intern_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
max_timestamp.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
metadata_watcher.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
metadata_watcher_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
queue_manager.go tsdb/wlog[PERF]: optimize WAL watcher reads (up to 540x less B/op; 13000x less allocs/op) (#18250) 2026-03-11 09:17:13 +00:00
queue_manager_test.go tsdb/wlog[PERF]: optimize WAL watcher reads (up to 540x less B/op; 13000x less allocs/op) (#18250) 2026-03-11 09:17:13 +00:00
read.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
read_handler.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
read_handler_test.go fix(remote_read): NHCB not returned over remote read samples (#17794) 2026-01-07 08:44:57 +01:00
read_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
stats.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
storage.go feat(storage)[PART4b]: add AppenderV2 to the rest of storage.Storage impl 2026-01-14 13:54:27 +00:00
storage_test.go Remove copyright date from headers (#17785) 2026-01-05 13:46:21 +01:00
write.go tsdb/wlog[PERF]: optimize WAL watcher reads (up to 540x less B/op; 13000x less allocs/op) (#18250) 2026-03-11 09:17:13 +00:00
write_handler.go fix: handle ErrTooOldSample as 400 Bad Request in OTLP and v2 histogram write paths 2026-02-14 02:49:49 -08:00
write_handler_test.go refactor: use Appender mock for tests (#17992) 2026-02-02 15:56:11 +00:00
write_otlp_handler.go fix: handle ErrTooOldSample as 400 Bad Request in OTLP and v2 histogram write paths 2026-02-14 02:49:49 -08:00
write_otlp_handler_test.go refactor: switch OTLP handler to AppendableV2 (#17996) 2026-02-03 16:44:40 +00:00
write_test.go refactor: move OTLP handler to separate file (#17990) 2026-02-02 14:36:00 +00:00