813 commits

Author	SHA1	Message	Date
Bartlomiej Plotka	d0db9b693c	Merge pull request #18217 from bragi92/main ... remote_write : Add Certificate support for ingesting data into an Azure Monitor Workspace	2026-06-08 09:55:53 +00:00
Kaveesh Dubey (from Dev Box)	f972cac551	azuread: import config_util used by CertificatePassword Secret type ... The previous commit changed CertificatePassword from string to config_util.Secret but did not add the corresponding import. The CI build for this PR alone passed only because GitHub builds the merge of the PR with upstream main, which already imports config_util (introduced in upstream commit 5ccebcdb3 for ClientSecret). Add the import so the PR's azuread.go is self-consistent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Kaveesh Dubey (from Dev Box) <kadubey@microsoft.com>	2026-06-01 21:36:19 -07:00
bragi92	933dd26f35	Update storage/remote/azuread/azuread.go ... Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: bragi92 <kadubey@microsoft.com>	2026-06-01 11:27:10 -07:00
Owen Williams	134051d480	tsdb: Add TODOs for ST-in-WAL work (#18773) ... Comment-only changes. This will make it easier for me to track my work. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2026-05-22 13:37:35 -04:00
bwplotka	1ae1172598	typo ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-05-08 10:18:42 +01:00
Bartlomiej Plotka	ef01f33a03	Apply suggestions from code review ... Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	2026-05-08 10:17:57 +01:00
bwplotka	38ccf6e7fe	fix: check bounds on remote write receive when parsing symbolized metadata ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-05-08 09:43:29 +01:00
Julien Pivotto	5e4c110c65	Merge branch 'release-3.11' into merge-3.11	2026-04-27 17:11:39 +02:00
Julien	26dae7f06b	Merge pull request #18590 from roidelapluie/roidelapluie/azadsecret ... remote/azuread: use Secret type for OAuth client_secret	2026-04-27 13:42:46 +02:00
Julien	0a1c5622d6	Merge pull request #18586 from roidelapluie/roidelapluie/azadsecret ... remote/azuread: use Secret type for OAuth client_secret	2026-04-27 12:48:28 +02:00
Julien Pivotto	5ccebcdb3f	remote/azuread: use Secret type for OAuth client_secret ... The ClientSecret field in OAuthConfig was typed as plain string, causing it to be exposed in plaintext via the /-/config HTTP endpoint. Change it to config_util.Secret so Prometheus redacts it as <secret>. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-04-27 12:17:49 +02:00
Julien Pivotto	3273935170	remote: validate snappy decoded length before allocation in read endpoint ... Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-04-27 12:02:50 +02:00
Ayoub Mrini	0dd834e924	Merge pull request #18406 from machine424/depll ... test: migrate TestDelayedCompaction to synctest to eliminate flakiness	2026-04-01 16:40:50 +02:00
Björn Rabenstein	4280662cdf	Merge pull request #18304 from crawfordxx/fix-typos-in-comments ... Fix typos in comments and metric help strings	2026-04-01 13:45:59 +02:00
Julien Pivotto	5d695516ba	storage/remote: add OTLP request body read limit ... Apply the same io.LimitReader guard (decodeReadLimit = 32 MiB) to the OTLP write decoder that remote read already use, so that a gzip-encoded request body cannot decompress to unbounded memory. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2026-03-31 11:11:47 +02:00
Copilot	bb8f8bdbca	Replace util/certutil with azidentity.ParseCertificates (#4) ... Address review feedback: use azidentity.ParseCertificates instead of the custom util/certutil package for certificate parsing. The azidentity package already handles PEM and PKCS#12 formats natively. - Remove util/certutil package entirely - Use azidentity.ParseCertificates in newCertificateTokenCredential - Revert golang.org/x/crypto to indirect dependency Agent-Logs-Url: https://github.com/bragi92/prometheus/sessions/efc3d6c5-9927-4d5b-8aa9-afe94b659c6e Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com>	2026-03-30 14:10:49 -07:00
machine424	dcfb8ce59c	chore: remove util/testutil/synctest now that we use Go>=1.25 ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2026-03-30 19:48:39 +02:00
crawfordxx	afaff7e116	Fix typos in comments and metric help strings ... - limt -> limit (storage/remote/queue_manager.go metric help text) - exluded -> excluded (tsdb/compact.go comment) - wont -> won't (tsdb/head.go comment) Signed-off-by: crawfordxx <crawfordxx@users.noreply.github.com>	2026-03-18 12:22:09 +08:00
bwplotka	3cf43337dc	post merge conflict fixes ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-03-12 09:03:08 +00:00
bwplotka	c133a969af	Merge branch 'main' into start-time-main-sync	2026-03-12 08:28:15 +00:00
Bartlomiej Plotka	a73202012b	tsdb/wlog[PERF]: optimize WAL watcher reads (up to 540x less B/op; 13000x less allocs/op) (#18250) ... See the detailed analysis https://docs.google.com/document/d/1efVAMcEw7-R_KatHHcobcFBlNsre-DoThVHI8AO2SDQ/edit?tab=t.0 I ran extensive benchmarks using synthetic data as well as real WAL segments pulled from the prombench runs. All benchmarks are here https://github.com/prometheus/prometheus/compare/bwplotka/wal-reuse?expand=1 * optimization(tsdb/wlog): reuse Ref* buffers across WAL watchers' reads Signed-off-by: bwplotka <bwplotka@gmail.com> * optimization(tsdb/wlog): avoid expensive error wraps Signed-off-by: bwplotka <bwplotka@gmail.com> * optimization(tsdb/wlog): reuse array for filtering Signed-off-by: bwplotka <bwplotka@gmail.com> * fmt Signed-off-by: bwplotka <bwplotka@gmail.com> * lint fix Signed-off-by: bwplotka <bwplotka@gmail.com> * tsdb/record: add test for clear() on histograms Signed-off-by: bwplotka <bwplotka@gmail.com> * updated WriteTo with what's currently expected Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-03-11 09:17:13 +00:00
Bartlomiej Plotka	d448f3f970	Merge pull request #18218 from prometheus/utilrecord ... tests(util/testwal): Move WAL record generation to separate package for reuse	2026-03-05 10:07:57 +01:00
Matthieu MOREL	45b9329e68	chore: fix emptyStringTest issues from gocritic (#18226) ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2026-03-04 08:24:50 +01:00
Kyle Eckhart	897ba10d11	remote write: fix sent_batch_duration_seconds measuring before the request is sent (#18214) ... * remote write: fix sent_batch_duration_seconds measuring before the request was sent Signed-off-by: Kyle Eckhart <kgeckhart@users.noreply.github.com>	2026-03-03 10:57:55 -08:00
bwplotka	596830ee72	tests(util/testwal): Move WAL record generation to separate package for reuse ... Signed-off-by: bwplotka <bwplotka@gmail.com> tmp Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-03-03 10:06:21 +00:00
Copilot	81c2ad71b4	Add Azure AD certificate-based authentication for remote write (#2) ... * Initial plan * Add Azure AD certificate-based authentication support Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Update documentation for certificate-based authentication Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Address code review feedback - improve error messages and format detection Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Replace third-party go-pkcs12 with official golang.org/x/crypto/pkcs12 Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Extract certificate parsing to common util/certutil package Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> * Fix linting errors: import ordering and comment formatting Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bragi92 <28612268+bragi92@users.noreply.github.com>	2026-03-02 13:21:42 -08:00
bwplotka	0ad8516ce0	fixed tests after rebase ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:15:22 +00:00
bwplotka	f27ca31bed	tests: add bench CLI recommended invokations ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:05:49 +00:00
bwplotka	c2eac549d5	tests: test ST in a cheapest way possible ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:05:49 +00:00
bwplotka	ba1b87f51f	feat: RW2 sending ST support ... Signed-off-by: bwplotka <bwplotka@gmail.com> tmp Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-25 19:01:45 +00:00
bwplotka	8f3a6020d8	Merge branch 'main' into st-main-sync2	2026-02-25 13:54:25 +00:00
Bartlomiej Plotka	333e0dc188	tests: reinforce and optimize queue_manager_test createTimeSeries (#18179) ... * tests: fix createTimeSeries so it does not create unnecessary load 16M samples 4k series Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments Signed-off-by: bwplotka <bwplotka@gmail.com> * Apply suggestions from code review Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2026-02-25 13:22:13 +00:00
bwplotka	56c46af6a6	Merge branch 'main' into st-f-main	2026-02-23 10:00:39 +00:00
Bartlomiej Plotka	1e2529f8d6	Merge pull request #18084 from veeceey/fix/issue-16645-otlp-too-old-sample ... fix: handle ErrTooOldSample as 400 in OTLP and v2 histogram write paths	2026-02-17 09:11:18 +00:00
Arve Knudsen	4fb6ce4d9a	fix: prevent stale exemplars leaking to histogram _sum/_count across data points (#18056) ... In addHistogramDataPoints, exemplars assigned to the +Inf bucket of one data point were carried over into the _sum and _count Append calls of the next data point via the shared appOpts. Clear appOpts.Exemplars at the start of each loop iteration to restore the nil-exemplar semantics that existed before the AppenderV2 migration. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-02-15 10:47:38 +01:00
Varun Chawla	e72bc1381c	fix: handle ErrTooOldSample as 400 Bad Request in OTLP and v2 histogram write paths ... The OTLP write handler and the PRW v2 histogram append path were missing ErrTooOldSample from their error type checks, causing these errors to fall through to the default case and return HTTP 500 Internal Server Error. This triggered unnecessary retries in OTLP clients like the Python SDK. The PRW v1 write handler (line 115) and the PRW v2 sample append path (line 377) already correctly handle ErrTooOldSample as a 400, and this change makes the remaining paths consistent. Also adds ErrTooOldSample to the v1 sample/histogram log checks so these errors are properly logged instead of silently returned. Fixes #16645 Signed-off-by: Varun Chawla <varun_6april@hotmail.com>	2026-02-14 02:49:49 -08:00
Owen Williams	b57f5b59b3	tsdb: ST-in-WAL: Counter implementation and benchmarks (#17671) ... Initial implementation of https://github.com/prometheus/prometheus/issues/17790. Only implements ST-per-sample for Counters. Tests and benchmarks updated. Note: This increases the size of the RefSample object for all users, whether st-per-sample is turned on or not. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2026-02-12 13:17:50 -05:00
Ganesh Vernekar	0ede19e2ee	Replace deprecated google.CredentialsFromJSON with option.WithAuthCredentialsFile ... Signed-off-by: Ganesh Vernekar <ganesh.vernekar@reddit.com>	2026-02-10 13:03:14 -08:00
Bartlomiej Plotka	5e46e77754	refactor: use Appender mock for otlptranslator tests (#17999) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-06 09:51:40 +00:00
Bartlomiej Plotka	7769495a4a	refactor: switch OTLP handler to AppendableV2 (#17996) ... * refactor: switch OTLP handler to AppendableV2 Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-03 16:44:40 +00:00
Bartlomiej Plotka	44d772b4e7	refactor: use Appender mock for tests (#17992) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-02 15:56:11 +00:00
Bartlomiej Plotka	076369fad0	refactor: move OTLP handler to separate file (#17990) ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2026-02-02 14:36:00 +00:00
Arve Knudsen	dc34b90f93	otlptranslator: fix silently swallowed error in addSumNumberDataPoints (#17954) ... The createAttributes error was incorrectly returning nil instead of err, causing errors to be silently discarded. This could lead to silent data loss for sum metrics during OTLP ingestion. Fixes #17953 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-01-28 13:58:50 +01:00
Callum Styan	97e7ef802c	remote write: simplify readability of timeseries filtering by using the slices package (#14318) ... * simplify readability of timeseries filtering by using the slices package Signed-off-by: Callum Styan <callumstyan@gmail.com> * ensure that BenchmarkBuildTimeSeries doesn't account for the building of the actual proto in the benchmark results, we only care about the buildTimeSeries call Signed-off-by: Callum Styan <callumstyan@gmail.com> --------- Signed-off-by: Callum Styan <callumstyan@gmail.com>	2026-01-28 11:47:34 +00:00
Arve Knudsen	2332962c4b	otlptranslator: filter __name__ from OTLP attributes to prevent duplicates (#17917) ... * otlptranslator: filter __name__ from OTLP attributes to prevent duplicates OTLP metrics can have a __name__ attribute which, when combined with the metric name passed via extras, creates duplicate __name__ labels. This commit implements filtering out of any __name__ metric attribute from OTLP. Also rename TestCreateAttributes to TestPrometheusConverter_createAttributes for consistency, and add test cases for __name__, __type__, and __unit__ OTLP metric attributes. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-01-24 16:44:08 +01:00
Ben Kochie	1a0c56993c	Use standard library errors in otlptranslator (#17894) ... Remove direct use of `go.uber.org/multierr` in favor of standard library `errors` package. Signed-off-by: SuperQ <superq@gmail.com>	2026-01-20 10:13:40 +01:00
Bartlomiej Plotka	c4b0da94db	Merge pull request #17835 from prometheus/bwplotka/a2-storage-support ... feat(storage)[PART4b]: add AppenderV2 to the rest of storage.Storage implementations + mock exemplar fix	2026-01-16 13:04:14 +00:00
Arve Knudsen	4afa76d083	OTLP: label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency (#17860) ... * otlptranslator: add label caching for OTLP-to-Prometheus conversion Add per-request caching to reduce redundant computation and allocations during OTLP metric conversion: 1. Per-request label sanitization cache: Cache sanitized label names within a request to avoid repeated string allocations for commonly repeated labels like __name__, job, instance. 2. Resource-level label caching: Precompute and cache job, instance, promoted resource attributes, and external labels once per ResourceMetrics boundary instead of for each datapoint. 3. Scope-level label caching: Precompute and cache scope metadata labels (otel_scope_name, otel_scope_version, etc.) once per ScopeMetrics boundary. 4. LabelNamer instance caching: Reuse the LabelNamer struct across datapoints within the same resource context. These optimizations significantly reduce allocations and improve latency for OTLP ingestion workloads with many datapoints per resource/scope. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2026-01-16 10:30:05 +00:00
George Krajcsovits	ef350b2b54	Merge pull request #17840 from prometheus/krajo/st-in-chunks ... feat(tsdb): new Append parameter and AtST interface for chunks	2026-01-16 10:32:53 +01:00
Arve Knudsen	a85a8998c9	otlptranslator: fix BenchmarkPrometheusConverter_FromMetrics (#17866) ... The benchmark was passing appendMetadata=false to NewCombinedAppender, which caused UpdateMetadata to never be called on the underlying noOpAppender. This resulted in app.metadata always being 0, failing the assertion that metadata count should be positive. Fix by enabling metadata appending in the benchmark. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2026-01-15 09:33:16 +01:00