* web/api: reject 0 for limit and batch_size in search endpoints
Treat 0 as invalid for limit and batch_size query parameters; clients
must supply a positive integer or omit the parameter to use the server
default of 100. Update OpenAPI descriptions accordingly.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Update web/api/v1/openapi.go
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Julien <291750+roidelapluie@users.noreply.github.com>
---------
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: Julien <291750+roidelapluie@users.noreply.github.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Add a mutex around the fgprof handler so that only one profile can
run at a time. A second concurrent request returns 500 with an error
message matching the pprof behaviour for CPU profiling already in use.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
The retention.percentage config field was typed as uint, which silently
truncated fractional values. Setting percentage: 1.5 in prometheus.yml
resulted in a retention of 1%, with no warning or error.
Remove the redundant MaxPercentage > 100 clamp in main.go; the config
UnmarshalYAML already returns an error for out-of-range values before
this code is reached.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Introduce DefaultTSDBRetentionConfig, populated from CLI flags before any
config file is loaded, so that retention falls back to CLI flags when the
config file has no storage.tsdb section. Config.UnmarshalYAML always injects
a non-nil TSDBConfig with those defaults, removing the need for nil checks in
main.go. ApplyConfig in web.go now propagates retention settings on each
config reload so the runtime info endpoint stays up to date.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Parser configuration is now per-engine/API/loader and no longer uses package-level flags, so behavior is consistent and tests don't rely on save/restore of global variables.
Signed-off-by: Martin Valiente Ainz <64830185+tinitiuset@users.noreply.github.com>
* Add OpenAPI 3.2 specification generation for Prometheus HTTP API
This commit introduces an OpenAPI specification for the Prometheus API.
After testing multiple code-generation servers with built-in APIs, this
implementation uses an independent spec file outside of the critical path.
This spec file is tested with a framework present in this pull request.
The specification helps clients know which parameters they can use and is
served at /api/v1/openapi.yaml. The spec file will evolve with the
Prometheus API and has the same version number.
Downstream projects can tune the APIs presented in the spec file with
configuration options using the IncludePaths setting for path filtering.
In the future, there is room to generate a server from this spec file
(e.g. with interfaces), but this is out of scope for this pull request.
Architecture:
- Core OpenAPI infrastructure (openapi.go): Dynamic spec building,
caching, and thread-safe spec generation
- Schema definitions (openapi_schemas.go): Complete type definitions
for all API request and response types
- Path specifications (openapi_paths.go): Endpoint definitions with
parameters, request bodies, and response schemas
- Examples (openapi_examples.go): Realistic request/response examples
- Helper functions (openapi_helpers.go): Reusable builders for common
OpenAPI structures
Testing:
- Comprehensive test suite with golden file validation
- Test helpers package for API testing infrastructure
- OpenAPI compliance validation utilities
The golden file captures the complete specification for snapshot testing.
Update with: go test -run TestOpenAPIGolden -update-openapi-spec
REVIEWERS: The most important thing to check would be the OpenAPI golden
file (web/api/v1/testdata/openapi_golden.yaml). Test scenarios are important
as they test the actual OpenAPI spec validity.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Add OpenAPI 3.1 support with version selection
Add support for both OpenAPI 3.1 and 3.2 specifications with version
selection via openapi_version query parameter. Defaults to 3.1 for
broader compatibility
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Enhance OpenAPI examples and add helper functions
- Add timestampExamples helper for consistent time formatting
- Add exampleMap helper to simplify example creation
- Improve example summaries with query details
- Add matrix result example for range vector queries
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* web/api: Add AtST method to test helper iterators
Implement the AtST() method required by chunkenc.Iterator interface
for FakeSeriesIterator and FakeHistogramSeriesIterator test helpers.
The method returns 0 as these test helpers don't use start timestamps
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* OpenAPI: Add minimum coverage test
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* OpenAPI: Improve examples handling
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---------
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
The /classic/static/* route was added to serve vendor JavaScript and CSS
files (jQuery, Bootstrap, etc.) for console templates. These vendor assets
were removed in #14807 due to security vulnerabilities, making this route
obsolete as it now serves an empty directory.
The console feature remains functional via --web.console.templates and
--web.console.libraries flags. Users who need JavaScript/CSS libraries in
their custom console templates must provide these assets within the
directory specified by --web.console.libraries.
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Partially fixes https://github.com/prometheus/prometheus/issues/17416 by
renaming all CT* names to ST* in the whole codebase except RW2 (this is
done in separate
[PR](https://github.com/prometheus/prometheus/pull/17411)) and
PrometheusProto exposition proto.
```
CreatedTimestamp -> StartTimestamp
CreatedTimeStamp -> StartTimestamp
created_timestamp -> start_timestamp
CT -> ST
ct -> st
```
Signed-off-by: bwplotka <bwplotka@gmail.com>
This reverts commit 93887a9b82.
See discussion at https://github.com/prometheus/prometheus/pull/17240 - I had
thought this was a simple and clear bugfix, but multiple people were depending
on the previous behavior already, and we want to err on the side of stability.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Currently the API always returns http code 422 for engine execution error, and
This PR allows the error code to be overriden, based on the ErrorType and the error itself.
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <justinjung04@gmail.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
* PROM-39: Add type and unit labels to OTLP endpoint
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Extract label addition into helper function
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Wire feature flag and web handler configuration
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Apply suggestions from code review
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Use lowercase for units too
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Use otlptranslator.UnitNamer to build units
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Address copilot's comment
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Verify label presence before adding them
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Overwrite type/unit labels when already set
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* sed/addTypeAndUnitLabels/enableTypeAndUnitLabels/
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Reduce duplicated code
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
---------
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
* OTLP receiver: Generate target_info samples between the earliest and latest samples per resource
Modify the OTLP receiver to generate target_info samples between the earliest
and latest samples per resource instead of only one for the latest timestamp.
The samples are spaced lookback delta/2 apart.
---------
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
What
Adds support for OTLP delta temporality to the OTLP endpoint.
This is done by calling the deltatocumulative processor from the OpenTelemetry collector during OTLP conversion.
Why
Delta conversion is a naturally stateful process, which requires careful request routing when operated inside a collector.
Prometheus is already stateful and doing the conversion in-server reduces the operational burden on the ingest architecture by only having one stateful component.
How
deltatocumulative is a OTel collector component that works as follows:
* pmetric.Metrics come from a receiver or in this case from the HTTP client
* It operates as an in-place update loop:
* for each sample, if not delta, leave unmodified
* if delta, do:
* state += sample, where state is the in-memory sum of all previous samples
* sample = state, sample value is now cumulative
* this is supported for sums (counters), gauges, histograms (old histograms) and exponential histograms (native histograms)
If a series receives no new samples for 5m, its state is removed from memory
Performance
Delta performance is a stateful operation and the OTel code is not highly optimized yet, e.g. it locks the entire processor for each request. Nonetheless, care has been taken to mitigate those effects:
delta conversion is behind a feature flag. If disabled, no conversion code is ever invoked
if enabled, conversion is not invoked if request not actually contains delta samples. This leads to no measureable performance difference between default-cumulative to convert-cumulative (only cumulative, feature on/off)
Signed-off-by: sh0rez <me@shorez.de>
This commit introduced two field in `/status` endpoint:
- The node currently serving the request.
- The current server time for debugging time drift issues.
fixes#15394.
Signed-off-by: sujal shah <sujalshah28092004@gmail.com>
For: #14355
This commit updates Prometheus to adopt stdlib's log/slog package in
favor of go-kit/log. As part of converting to use slog, several other
related changes are required to get prometheus working, including:
- removed unused logging util func `RateLimit()`
- forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger
- move some of the json file logging functionality to use prom/common package functionality
- refactored some of the new json file logging for scraping
- changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers
- updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition
- added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type
Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>
This commit introduces a new `/api/v1/notifications/live` endpoint that
utilizes Server-Sent Events (SSE) to stream notifications to the web UI.
This is used to display alerts such as when a configuration reload
has failed.
I opted for SSE over WebSockets because SSE is simpler to implement and
more robust for our use case. Since we only need one-way communication
from the server to the client, SSE fits perfectly without the overhead
of establishing and maintaining a two-way WebSocket connection.
When the SSE connection fails, we go back to a classic
/api/v1/notifications API endpoint.
This commit also contains the required UI changes for the new Mantine UI.
Signed-off-by: Julien <roidelapluie@o11y.eu>
