prometheus/documentation/examples/custom-sd
Will Bollock b70a871988
fix(discovery): delete expired refresh metrics on reload (#17614)
Building off config-specific Prometheus refresh metrics from an earlier
PR (https://github.com/prometheus/prometheus/pull/17138), this deletes
refresh metrics like `prometheus_sd_refresh_duration_seconds` and
`prometheus_sd_refresh_failures_total` when the underlying scrape job
configuration is removed on reload. This reduces un-needed cardinality
from scrape job specific metrics while still preserving metrics that
indicate overall health of a service discovery engine.

For example,
`prometheus_sd_refresh_failures_total{config="linode-servers",mechanism="linode"} 1`
will no longer be exported by Prometheus when the `linode-servers`
scrape job for the Linode service provider is removed. The generic,
service discovery specific `prometheus_sd_linode_failures_total` metric
will persist however.

* fix: add targetsMtx lock for targets access

* test: validate refresh/discover metrics are gone

* ref: combine sdMetrics and refreshMetrics

Good idea from @bboreham to combine sdMetrics and refreshMetrics!
They're always passed around together and don't have much of a
reason not to be combined. mechanismMetrics makes it clear what kind of
metrics this is used for (service discovery mechanisms).

---------

Signed-off-by: Will Bollock <wbollock@linode.com>
2026-04-02 13:43:35 +01:00
..
adapter fix(discovery): delete expired refresh metrics on reload (#17614) 2026-04-02 13:43:35 +01:00
adapter-usage fix(discovery): delete expired refresh metrics on reload (#17614) 2026-04-02 13:43:35 +01:00
README.md Fix typo (#4760) 2018-10-18 21:19:21 +01:00

Custom SD

Custom SD is available via the file_sd adapter, which can be used to access SD mechanisms that are not included in official Prometheus releases. The sd adapter outputs a file that can be passed via your prometheus.yml as a file_sd file. This will allow you to pass your targets to Prometheus for scraping without having to use a static config.

Example

This directory (documentation/examples/custom-sd) contains an example custom service discovery implementation as well as a file_sd adapter usage. adapter-usage contains the Discoverer implementation for a basic Consul service discovery mechanism. It simply queries Consul for all it's known services (except Consul itself), and sends them along with all the other service data as labels as a TargetGroup. The adapter directory contains the adapter code you will want to import and pass your Discoverer implementation to.

Usage

To use file_sd adapter you must implement a Discoverer. In adapter-usage/main.go replace the example SD config and create an instance of your SD implementation to pass to the Adapter's NewAdapter. See the Note: comments for the structs and Run function.