1771 commits

Author	SHA1	Message	Date
Bryan Boreham	0f75a307f1	Merge pull request #18860 from texasich/fix/flaky-reshard-partial-batch ... storage/remote: don't run TestReshardPartialBatch in parallel	2026-06-09 12:17:47 +01:00
Bartlomiej Plotka	d0db9b693c	Merge pull request #18217 from bragi92/main ... remote_write : Add Certificate support for ingesting data into an Azure Monitor Workspace	2026-06-08 09:55:53 +00:00
texasich	8c270501fc	storage/remote: don't run TestReshardPartialBatch in parallel ... Fixes #18078 `TestReshardPartialBatch` runs 100 iterations of `shards.stop()` / `shards.start(1)` with a 2-second deadline per iteration, treating any slower completion as a deadlock. Under normal conditions one iteration takes ~10–50ms, so the 2s window is plenty. But the stack trace attached to #18078 shows the test timing out while 72 `runShard` goroutines from other parallel tests are active — those are from `TestReshard` (spawns up to 60 shards) and `TestReshardRaceWithStop` (drives 100 reshards back-to-back). Under that kind of scheduler pressure, a legitimate non-deadlocked `stop()` can occasionally cross 2s, and the test misreports it as a deadlock. `t.Parallel()` was added to `TestReshardPartialBatch` (and most of its neighbours) in bulk in fe1bb5337 as part of a general "parallelize everything" pass — the flakiness started from that point. The other reshard tests are less timing-sensitive, but this one explicitly measures shutdown latency, so running it alongside them invalidates the premise. Dropping `t.Parallel()` here restores the test's original isolation. It still detects a real deadlock (it would never complete), it just stops firing false positives when the CPU is saturated by its siblings. ```release-notes NONE ``` Signed-off-by: texasich <101962694+texasich@users.noreply.github.com>	2026-06-04 19:29:19 -05:00
Kaveesh Dubey (from Dev Box)	f972cac551	azuread: import config_util used by CertificatePassword Secret type ... The previous commit changed CertificatePassword from string to config_util.Secret but did not add the corresponding import. The CI build for this PR alone passed only because GitHub builds the merge of the PR with upstream main, which already imports config_util (introduced in upstream commit 5ccebcdb3 for ClientSecret). Add the import so the PR's azuread.go is self-consistent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Kaveesh Dubey (from Dev Box) <kadubey@microsoft.com>	2026-06-01 21:36:19 -07:00
bragi92	933dd26f35	Update storage/remote/azuread/azuread.go ... Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: bragi92 <kadubey@microsoft.com>	2026-06-01 11:27:10 -07:00
Owen Williams	134051d480	tsdb: Add TODOs for ST-in-WAL work (#18773) ... Comment-only changes. This will make it easier for me to track my work. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2026-05-22 13:37:35 -04:00
Julien	e1f4380b2a	web/api: add search API endpoint (#18573) ... Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-05-19 13:58:00 +02:00
bwplotka	1ae1172598	typo ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-05-08 10:18:42 +01:00
Bartlomiej Plotka	ef01f33a03	Apply suggestions from code review ... Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	2026-05-08 10:17:57 +01:00
bwplotka	38ccf6e7fe	fix: check bounds on remote write receive when parsing symbolized metadata ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-05-08 09:43:29 +01:00
Julien Pivotto	5e4c110c65	Merge branch 'release-3.11' into merge-3.11	2026-04-27 17:11:39 +02:00
Julien	26dae7f06b	Merge pull request #18590 from roidelapluie/roidelapluie/azadsecret ... remote/azuread: use Secret type for OAuth client_secret	2026-04-27 13:42:46 +02:00
Julien	0a1c5622d6	Merge pull request #18586 from roidelapluie/roidelapluie/azadsecret ... remote/azuread: use Secret type for OAuth client_secret	2026-04-27 12:48:28 +02:00
Julien Pivotto	5ccebcdb3f	remote/azuread: use Secret type for OAuth client_secret ... The ClientSecret field in OAuthConfig was typed as plain string, causing it to be exposed in plaintext via the /-/config HTTP endpoint. Change it to config_util.Secret so Prometheus redacts it as <secret>. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-04-27 12:17:49 +02:00
Julien Pivotto	3273935170	remote: validate snappy decoded length before allocation in read endpoint ... Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-04-27 12:02:50 +02:00
Julien Pivotto	f69db5bc54	storage: introduce search interface with scoring and filtering ... Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-04-23 15:05:48 +02:00
Ayoub Mrini	0dd834e924	Merge pull request #18406 from machine424/depll ... test: migrate TestDelayedCompaction to synctest to eliminate flakiness	2026-04-01 16:40:50 +02:00
Björn Rabenstein	4280662cdf	Merge pull request #18304 from crawfordxx/fix-typos-in-comments ... Fix typos in comments and metric help strings	2026-04-01 13:45:59 +02:00
Julien Pivotto	5d695516ba	storage/remote: add OTLP request body read limit ... Apply the same io.LimitReader guard (decodeReadLimit = 32 MiB) to the OTLP write decoder that remote read already use, so that a gzip-encoded request body cannot decompress to unbounded memory. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-03-31 11:11:47 +02:00
Copilot	bb8f8bdbca	Replace util/certutil with azidentity.ParseCertificates (#4) ... Address review feedback: use azidentity.ParseCertificates instead of the custom util/certutil package for certificate parsing. The azidentity package already handles PEM and PKCS#12 formats natively. - Remove util/certutil package entirely - Use azidentity.ParseCertificates in newCertificateTokenCredential - Revert golang.org/x/crypto to indirect dependency Agent-Logs-Url: https://github.com/bragi92/prometheus/sessions/efc3d6c5-9927-4d5b-8aa9-afe94b659c6e Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com>	2026-03-30 14:10:49 -07:00
machine424	dcfb8ce59c	chore: remove util/testutil/synctest now that we use Go>=1.25 ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2026-03-30 19:48:39 +02:00
crawfordxx	afaff7e116	Fix typos in comments and metric help strings ... - limt -> limit (storage/remote/queue_manager.go metric help text) - exluded -> excluded (tsdb/compact.go comment) - wont -> won't (tsdb/head.go comment) Signed-off-by: crawfordxx <crawfordxx@users.noreply.github.com>	2026-03-18 12:22:09 +08:00
bwplotka	3cf43337dc	post merge conflict fixes ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-03-12 09:03:08 +00:00
bwplotka	c133a969af	Merge branch 'main' into start-time-main-sync	2026-03-12 08:28:15 +00:00
Bartlomiej Plotka	a73202012b	tsdb/wlog[PERF]: optimize WAL watcher reads (up to 540x less B/op; 13000x less allocs/op) (#18250) ... See the detailed analysis https://docs.google.com/document/d/1efVAMcEw7-R_KatHHcobcFBlNsre-DoThVHI8AO2SDQ/edit?tab=t.0 I ran extensive benchmarks using synthetic data as well as real WAL segments pulled from the prombench runs. All benchmarks are here https://github.com/prometheus/prometheus/compare/bwplotka/wal-reuse?expand=1 * optimization(tsdb/wlog): reuse Ref* buffers across WAL watchers' reads Signed-off-by: bwplotka <bwplotka@gmail.com> * optimization(tsdb/wlog): avoid expensive error wraps Signed-off-by: bwplotka <bwplotka@gmail.com> * optimization(tsdb/wlog): reuse array for filtering Signed-off-by: bwplotka <bwplotka@gmail.com> * fmt Signed-off-by: bwplotka <bwplotka@gmail.com> * lint fix Signed-off-by: bwplotka <bwplotka@gmail.com> * tsdb/record: add test for clear() on histograms Signed-off-by: bwplotka <bwplotka@gmail.com> * updated WriteTo with what's currently expected Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-03-11 09:17:13 +00:00
Bartlomiej Plotka	d448f3f970	Merge pull request #18218 from prometheus/utilrecord ... tests(util/testwal): Move WAL record generation to separate package for reuse	2026-03-05 10:07:57 +01:00
Matthieu MOREL	45b9329e68	chore: fix emptyStringTest issues from gocritic (#18226) ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2026-03-04 08:24:50 +01:00
Kyle Eckhart	897ba10d11	remote write: fix sent_batch_duration_seconds measuring before the request is sent (#18214) ... * remote write: fix sent_batch_duration_seconds measuring before the request was sent Signed-off-by: Kyle Eckhart <kgeckhart@users.noreply.github.com>	2026-03-03 10:57:55 -08:00
bwplotka	596830ee72	tests(util/testwal): Move WAL record generation to separate package for reuse ... Signed-off-by: bwplotka <bwplotka@gmail.com> tmp Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-03-03 10:06:21 +00:00
Copilot	81c2ad71b4	Add Azure AD certificate-based authentication for remote write (#2) ... * Initial plan * Add Azure AD certificate-based authentication support Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Update documentation for certificate-based authentication Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Address code review feedback - improve error messages and format detection Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Replace third-party go-pkcs12 with official golang.org/x/crypto/pkcs12 Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Extract certificate parsing to common util/certutil package Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Fix linting errors: import ordering and comment formatting Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com>	2026-03-02 13:21:42 -08:00
bwplotka	0ad8516ce0	fixed tests after rebase ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:15:22 +00:00
bwplotka	f27ca31bed	tests: add bench CLI recommended invokations ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:05:49 +00:00
bwplotka	c2eac549d5	tests: test ST in a cheapest way possible ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:05:49 +00:00
bwplotka	ba1b87f51f	feat: RW2 sending ST support ... Signed-off-by: bwplotka <bwplotka@gmail.com> tmp Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:01:45 +00:00
bwplotka	8f3a6020d8	Merge branch 'main' into st-main-sync2	2026-02-25 13:54:25 +00:00
Bartlomiej Plotka	333e0dc188	tests: reinforce and optimize queue_manager_test createTimeSeries (#18179) ... * tests: fix createTimeSeries so it does not create unnecessary load 16M samples 4k series Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments Signed-off-by: bwplotka <bwplotka@gmail.com> * Apply suggestions from code review Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2026-02-25 13:22:13 +00:00
bwplotka	56c46af6a6	Merge branch 'main' into st-f-main	2026-02-23 10:00:39 +00:00
George Krajcsovits	223f016c44	feat(tsdb): allow using ST capable XOR chunks - retain format on read (#18013) ... * feat(tsdb): allow appending to ST capable XOR chunk optionally Only for float samples as of now. Supports for in-order and out-of-order samples. Make sure that on readout the ST capable chunks are returned automatically. When the chunks are returned as is, this is trivially true. When a chunk needs to be re-coded due to deletion (tombstone) markers, we take the encoding of the original chunk. When a chunk needs to be created from overlapping chunks, we observe whether ST is zero or not and create the new chunk based on that. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2026-02-20 09:15:51 +01:00
Bartlomiej Plotka	1e2529f8d6	Merge pull request #18084 from veeceey/fix/issue-16645-otlp-too-old-sample ... fix: handle ErrTooOldSample as 400 in OTLP and v2 histogram write paths	2026-02-17 09:11:18 +00:00
Arve Knudsen	4fb6ce4d9a	fix: prevent stale exemplars leaking to histogram _sum/_count across data points (#18056) ... In addHistogramDataPoints, exemplars assigned to the +Inf bucket of one data point were carried over into the _sum and _count Append calls of the next data point via the shared appOpts. Clear appOpts.Exemplars at the start of each loop iteration to restore the nil-exemplar semantics that existed before the AppenderV2 migration. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-02-15 10:47:38 +01:00
Varun Chawla	e72bc1381c	fix: handle ErrTooOldSample as 400 Bad Request in OTLP and v2 histogram write paths ... The OTLP write handler and the PRW v2 histogram append path were missing ErrTooOldSample from their error type checks, causing these errors to fall through to the default case and return HTTP 500 Internal Server Error. This triggered unnecessary retries in OTLP clients like the Python SDK. The PRW v1 write handler (line 115) and the PRW v2 sample append path (line 377) already correctly handle ErrTooOldSample as a 400, and this change makes the remaining paths consistent. Also adds ErrTooOldSample to the v1 sample/histogram log checks so these errors are properly logged instead of silently returned. Fixes #16645 Signed-off-by: Varun Chawla <varun_6april@hotmail.com>	2026-02-14 02:49:49 -08:00
Owen Williams	b57f5b59b3	tsdb: ST-in-WAL: Counter implementation and benchmarks (#17671) ... Initial implementation of https://github.com/prometheus/prometheus/issues/17790. Only implements ST-per-sample for Counters. Tests and benchmarks updated. Note: This increases the size of the RefSample object for all users, whether st-per-sample is turned on or not. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2026-02-12 13:17:50 -05:00
Ganesh Vernekar	0ede19e2ee	Replace deprecated google.CredentialsFromJSON with option.WithAuthCredentialsFile ... Signed-off-by: Ganesh Vernekar <ganesh.vernekar@reddit.com>	2026-02-10 13:03:14 -08:00
Bartlomiej Plotka	5e46e77754	refactor: use Appender mock for otlptranslator tests (#17999) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-06 09:51:40 +00:00
Bartlomiej Plotka	c8e7f4e2a6	tests: Unify TestDiskFillingUpAfterDisablingOOO and avoid hiding errors (#18017) ... * tests: Unify TestDiskFillingUpAfterDisablingOOO and avoid hiding errors Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-05 16:11:35 +00:00
Bartlomiej Plotka	7769495a4a	refactor: switch OTLP handler to AppendableV2 (#17996) ... * refactor: switch OTLP handler to AppendableV2 Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-03 16:44:40 +00:00
Arve Knudsen	3c44ca757d	storage: fix ToError() to return nil for empty partial errors (#18002) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-02-03 16:58:02 +01:00
Bartlomiej Plotka	44d772b4e7	refactor: use Appender mock for tests (#17992) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-02 15:56:11 +00:00
Bartlomiej Plotka	076369fad0	refactor: move OTLP handler to separate file (#17990) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-02 14:36:00 +00:00
Bartlomiej Plotka	9657c23c37	storage: optimized fanoutAppenderV2 (#17976) ... * storage: add BenchmarkFanoutAppenderV2 Signed-off-by: bwplotka <bwplotka@gmail.com> * fix: optimized fanoutAppenderV2 Signed-off-by: bwplotka <bwplotka@gmail.com> * optimized more Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-02 07:04:30 +00:00