From 6f640bd04284922e2aa8aa530675b470de412869 Mon Sep 17 00:00:00 2001 From: Carrie Edwards Date: Tue, 28 Apr 2026 11:00:55 -0700 Subject: [PATCH] Update documentation Signed-off-by: Carrie Edwards --- docs/feature_flags.md | 5 +- tsdb/docs/format/chunks.md | 100 ++++++++++++++++++++++++++++++++++++- 2 files changed, 102 insertions(+), 3 deletions(-) diff --git a/docs/feature_flags.md b/docs/feature_flags.md index 7771e02f6d..88c98fac17 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -98,7 +98,7 @@ Besides enabling this feature in Prometheus, start timestamps need to be exposed > * It introduces new WAL record type (SamplesV2) that can only be replayed with Prometheus 3.11 or later versions. > * For persistent storage support (TSDB blocks), you need to manually opt-in for XOR2 chunk format ([`xor2-encoding` flag](#xor2-chunk-encoding)). > This might change later once we finish experimentation phase with XOR2. -> * ST for native histograms and NHCBs are not yet implemented (see [#18315](https://github.com/prometheus/prometheus/issues/18315)). +> * ST for native histograms and NHCBs are not yet implemented end-to-end. The `histogramST` and `floathistogramST` chunk encodings (also gated behind [`xor2-encoding`](#xor2-chunk-encoding)) but other areas are still in progress (see [#18315](https://github.com/prometheus/prometheus/issues/18315)). > * PromQL use of ST is out of scope of this feature. ## Start timestamp (ST) usage in PromQL functions @@ -348,7 +348,8 @@ For more details, see the [proposal](https://github.com/prometheus/proposals/pul > * We are still experimenting on the final encoding. As of now this encoding can change in any Prometheus version. All your persistent block data will be lost between versions. > * This is encoding is new, meaning downstream tools and LTS systems might now support it yet (e.g. Thanos sidecar uploaded blocks). -This setting enables the new XOR2 chunk encoding for float samples, which provides better disk compression than the default XOR encoding for typical Prometheus workloads. This format also allow storing Start Timestamp (ST). +This setting enables the new XOR2 chunk encoding for float samples, which provides better disk compression than the default XOR encoding for typical Prometheus workloads. This format also allows storing Start Timestamp (ST). +It also enables the new `histogramST` and `floathistogramST` chunk encodings for histogram and float histogram samples, which extend the histogram chunk format with the same ST header and per-sample ST encoding. ## Extended Range Selectors diff --git a/tsdb/docs/format/chunks.md b/tsdb/docs/format/chunks.md index 32538d436b..bc4b7a136d 100644 --- a/tsdb/docs/format/chunks.md +++ b/tsdb/docs/format/chunks.md @@ -36,7 +36,7 @@ in-file offset (lower 4 bytes) and segment sequence number (upper 4 bytes). Notes: * `len`: Chunk size in bytes. 1 to 5 bytes long using the [`` encoding](https://go.dev/src/encoding/binary/varint.go). -* `encoding`: Currently either `XOR`, `histogram`, or `floathistogram`, see [code for numerical values](https://github.com/prometheus/prometheus/blob/02d0de9987ad99dee5de21853715954fadb3239f/tsdb/chunkenc/chunk.go#L28-L47). +* `encoding`: Currently one of `XOR`, `XOR2`, `histogram`, `floathistogram`, `histogramST`, or `floathistogramST`, see [code for numerical values](https://github.com/prometheus/prometheus/blob/02d0de9987ad99dee5de21853715954fadb3239f/tsdb/chunkenc/chunk.go#L28-L47). The `XOR2`, `histogramST`, and `floathistogramST` encodings extend their non-ST counterparts with optional Start Timestamp (ST) data and are gated behind the experimental [`xor2-encoding` feature flag](../../../docs/feature_flags.md#xor2-chunk-encoding). * `data`: See below for each encoding. * `checksum`: Checksum of `encoding` and `data`. It's a [cyclic redundancy check](https://en.wikipedia.org/wiki/Cyclic_redundancy_check) with the Castagnoli polynomial, serialised as an unsigned 32 bits big endian number. Can be referred as a `CRC-32C`. @@ -267,6 +267,67 @@ the encoding will therefore result in a short varbit representation. The upper bound of 33554430 is picked so that the varbit encoded value will take at most 4 bytes. +## Histogram ST chunk data + +The histogram ST chunk extends the histogram chunk format with optional Start +Timestamp (ST) data, using the same ST encoding scheme as XOR2. The base histogram +sample encoding is identical to the [histogram chunk](#histogram-chunk-data). +A 1-byte `st_header` is inserted after `histogram_flags`, and an optional ST +field is appended after each sample's data. + +``` +┌──────────────────────┬──────────────────────────┬────────────────────┬───────────────────────────────┬─────────────────────┬──────────────────┬──────────────────┬──────────────────────┬────────────────┬──────────────────┐ +│ num_samples │ histogram_flags <1 byte> │ st_header <1 byte> │ zero_threshold <1 or 9 bytes> │ schema │ pos_spans │ neg_spans │ custom_values │ samples │ padding │ +└──────────────────────┴──────────────────────────┴────────────────────┴───────────────────────────────┴─────────────────────┴──────────────────┴──────────────────┴──────────────────────┴────────────────┴──────────────────┘ +``` + +### Start timestamp encoding + +The ST header is one byte: + + ``` + ┌───────────────────────┬───────────────────────┐ + │ first_st_known<1 bit> │ st_changed_on<7 bits> │ + └───────────────────────┴───────────────────────┘ + ``` + +where the highest bit `first_st_known` indicates if `st_0` is present or not. +If the lower 7bits `st_changed_on` is 0, no `st_i (i>0)` is present. +Otherwise `st_i (i>=st_changed_on>)` is present, while +`st_i (0` if present. +* `st_1` is encoded as a `` delta from the timestamp of the + previous sample (or from 0 if not previously set). +* `st_i` (i > 1) is encoded as a `` "delta of delta" of the ST + difference from the previous sample. + + +### Samples data: + +Each `sample_i ` payload uses the exact same encoding as the equivalent +sample in the [histogram chunk](#histogram-chunk-data) (sample 0, sample 1, and +sample 2-and-following), but with an optional ST field. If an ST payload is present, +it is appended immediately after the histogram same payload. + +``` +┌─────────────────┬──────────────────────┐ +│ sample_0 │ ?st_0 │ +├─────────────────┼──────────────────────┤ +│ sample_1 │ ?st_1 │ +├─────────────────┼──────────────────────┤ +│ sample_2 │ ?st_2 │ +├─────────────────┼──────────────────────┤ +│ ... │ ... │ +├─────────────────┼──────────────────────┤ +│ sample_n │ ?st_n │ +└─────────────────┴──────────────────────┘ +``` + ## Float histogram chunk data Float histograms have the same layout as histograms apart from the encoding of samples. @@ -310,3 +371,40 @@ Float histograms have the same layout as histograms apart from the encoding of s │ ts_dod │ count_xor │ zero_count_xor │ sum_xor │ pos_bucket_0_xor │ ... │ pos_bucket_n_xor │ neg_bucket_0_xor │ ... │ neg_bucket_n_xor │ └─────────────────────┴────────────────────────┴─────────────────────────────┴──────────────────────┴───────────────────────────────┴─────┴───────────────────────────────┴───────────────────────────────┴─────┴───────────────────────────────┘ ``` + +## Float histogram ST chunk data + +The float histogram ST chunk extends the float histogram chunk format with +optional Start Timestamp (ST) data, with the same ST encoding scheme as +[Histogram ST](#histogram-st-chunk-data). THe float histogram sample encoding is unchanged. A 1-byte +`st_header` is inserted after `histogram_flags`, and an optional ST field is +appended after each sample's data. + +``` +┌──────────────────────┬──────────────────────────┬────────────────────┬───────────────────────────────┬─────────────────────┬──────────────────┬──────────────────┬──────────────────────┬────────────────┬──────────────────┐ +│ num_samples │ histogram_flags <1 byte> │ st_header <1 byte> │ zero_threshold <1 or 9 bytes> │ schema │ pos_spans │ neg_spans │ custom_values │ samples │ padding │ +└──────────────────────┴──────────────────────────┴────────────────────┴───────────────────────────────┴─────────────────────┴──────────────────┴──────────────────┴──────────────────────┴────────────────┴──────────────────┘ +``` + +### Samples data: + +Each `sample_i ` payload uses the exact same encoding as the equivalent +sample in the [float histogram chunk](#float-histogram-chunk-data) (sample 0, +sample 1, and sample 2-and-following). The optional ST field and the +`st_header` byte follow the same encoding rules as the +[histogram ST chunk](#histogram-st-chunk-data) and +[XOR2 start timestamp encoding](#start-timestamp-encoding). + +``` +┌─────────────────┬──────────────────────┐ +│ sample_0 │ ?st_0 │ +├─────────────────┼──────────────────────┤ +│ sample_1 │ ?st_1 │ +├─────────────────┼──────────────────────┤ +│ sample_2 │ ?st_2 │ +├─────────────────┼──────────────────────┤ +│ ... │ ... │ +├─────────────────┼──────────────────────┤ +│ sample_n │ ?st_n │ +└─────────────────┴──────────────────────┘ +```